Advise on Building an Assessment Plan from Dean Maryann Jones

More good stuff from the Building an Assessment Plan conference today: Maryann Jones, Dean Emerita, Western State College of Law, and Educational Consultant, spoke of the “paradigm shift” that legal education faces from a focus on teaching and faculty to a focus on student learning and accomplishments. Dean Jones related her experience in having to make that shift early on because her law school was an independent free-standing law school accredited by the regional accreditor and so was not “given a pass” on assessment as are many law schools that are part of larger university systems. She noted, however, that increasingly regional accreditors are paying attention to law schools, with more than one accreditation report noting that “The law school lags substantially behind the rest of the institution in terms of assessment of student learning.”

She provided a list of very helpful hints in building an assessment plan:

  • The plan has to be an integral part of the whole institution, not housed in an “assessment office” and not only in the curriculum, but must also include student services and co-curricular programs.
  • There must be an institutional commitment of resources
  • The faculty must own assessment. The plan must be built from the ground up, from the beginning, led by the faculty. You cannot simply hand the faculty a plan and tell them to “implement it” She emphasized the importance of one-on-one and small group conversations to develop that plan.
  • One size does not fit all. An assessment plan cannot be “plug and play” but must address its own mission and within its own resources and culture. On the other hand, you don’t have to reinvent the wheel. There are many good models that can provide a starting point.
  • Be creative. Work with what you have.
  • Do not make too many learning outcomes. Keep it manageable.
  • Get training. Regional accreditation programs often provide good training. There are great books available.
  • When you start the dialogue on assessment, bring in a facilitator who is familiar with assessment in the law school context.
  • Do not underestimate student involvement. Talk about student learning outcomes in classes. Widely disseminate your program learning outcomes.
  • Include institutional research – assessment is data driven, so you have to determine how you are going to gather that data from the beginning.
  • Close the loop on assessment. How will you use your results. Is it on the agenda of faculty meetings? Do you have quality improvement or action plan at the end of an assessment cycle? Do you regularly conduct program review (comprehensive review periodically)?

Thank you Maryann!

Resources from the Building an Assessment Plan Conference

In the morning session of the conference on Building an Assessment Plan from the Ground Up, presenters shared an overview of assessment in action.

Professor Andrea Curcio, Professor of Law and Co-director Externship Program at Georgia State University College of Law, presented on “The Purposes of Assessment” by providing two powerful examples of how assessment challenges faculty to change and improve their teaching. The first of these targeted a learning outcome of developing cultural sensitivity. She described challenging survey results from her law school regarding student ability to perceive their own cultural lens and the teaching challenges those results presented. The second was the learning outcome of analytical reasoning and her efforts in her civil procedure class to improve student skill through formative feedback. For both of these, Andrea has published articles that provide resources and analysis of these assessments.

Professor Vicki VanZandt of the University of Dayton School of Law and Professor Kelley Mauerman of Whittier Law School presented a primer on “Assessment 101” including much of the language of assessment, providing examples from their own schools off their curricular planning process.  They emphasized the importance of starting small and building an assessment plan from activities faculty already engage in. They reminded the audience that not every outcome needs to be assessed every year, providing examples of their law school’s own curriculum maps and assessment plans. Read the learning outcomes and performance criteria from Dayton here: and from Whittier here:

Professor VanZandt is the co-author of Student Learning Outcomes and Law School Assessment: A Practical Guide to Measuring Institutional Effectiveness (Carolina Academic Press 2015) available for purchase here:

More resources will be available, including slideshows and handouts from the conference.  Keep watching!

Building an Assessment Plan Conference off to a great start!

The conference on “Building an Assessment Plan from the Ground Up” is being held today at Whittier Law School. Professor Andrea Funk, Associate Dean for Lawyering Skills and Institutional Assessment
Professor of Lawyering Skills at Whittier, has created a comprehensive program for the sold-out crowd of faculty, most of whom have some responsibility for assessment planning at their law schools. For your use and enjoyment, I will be blogging from the conference today. Professor Judith Wegner,  Burton Craige Professor of Law at the University of North Carolina School of Law, began asking us to all think about words that begin with “A” and reflect on how many of those are disagreeable: aspirin, ache, anger, accreditation, assessment! She provided a powerful overview of three of these “A-words” that many faculty find very foreign: pressures for accountability in accreditation processes that require assessment of student learning outcomes. You can read more about the history and context of accreditation here:  .  Download her slideshow here (scroll to “Assessment”).


Teaching Legal Reasoning More Efficiently?

Teaching the traditional analytical skills more efficiently and effectively could provide a much needed opening for broadening the range of skills taught to all law students. In the legal academy’s version of the “socratic method”, law teachers historically taught the analytical skills” implicitly”. They demonstrated legal reasoning by pushing students away from their raw intuitions of fairness and justice to articulate rules and exceptions, while attending carefully to the inevitable ambiguities of language.

Some law teachers suggest that the process of learning to “think like a lawyer” fundamentally requires time and practice and therefore cannot be significantly speeded up.

Yet the implicit approach has been repeatedly challenged by scholars seeking to teach legal reasoning more explicitly, by naming and explaining how it works.*  (An obsession with the goal of teaching legal reasoning more efficiently was a major thread in two phases of my own legal career when I taught first year civil procedure. I struggled both to teach skills more explicitly and to provide students with opportunities to practice them.)

A recent contribution to this quest by my colleague Jane Winn grows out of her experiment teaching common law legal reasoning to undergraduates. Students were randomly assigned to use either a well-regarded study aid, or Winn’s own materials. The materials were also leavened by her own and colleagues’ experiences teaching foreign LL.M. and J.D. students coming from legal systems growing out of the European continental legal tradition.

Winn’s effort, aimed at law students, is notable in three respects. First, at twenty-nine pages it fills an intermediate-length niche: longer than a typical class “handout’, but shorter than the various book length alternatives. Second, it covers case briefing, outlining and exam questions, demonstrating how the three are related. Third, it grew out of an attempt to test her teaching method empirically using random assignment to a control group. Both law students and legal educators should find it a useful contribution.

The 2015 ABA accreditation standards may provide a laboratory in which to test efforts such as Winn’s. Standard 302 now requires law schools to adopt learning outcomes that, under subsection (b), must include legal analysis and reading; Standard 314 requires law schools to provide students with both formative assessment (feedback) and summative assessments (final “grades”); under Standard 315 law schools must engage in “ongoing evaluation of the program of education, learning outcomes, and assessment methods”. At its best this combination of more intentionally articulated outcomes, feedback to students, and program evaluation could prompt law schools to evaluate the potential for greater efficiency and effectiveness in teaching legal reasoning. I remain hopeful that enough schools will approach this task rigorously and in good faith that at least some progress can be made.

*Winn’s illustrious predecessors include:

  • Leading Legal Realist Karl Llewelyn, whose The Bramble Bush: Classic Lectures on Law and Law School have been assigned to generations of law students;
  • University of Chicago Professor and President and U.S. Attorney General Edward H. Levi, author of An Introduction to Legal Reasoning, originally published in the University of Chicago Law Review and then in book form;
  • Critical Theorist and Harvard Professor Duncan Kennedy, who took the decidedly un-Harvard step of visiting at New England School of Law in his attempt to reach beyond elite students and sharpen his skill at teaching students about the “gaps, conflicts and ambiguities” that underlie the development of the common law. He shared his insights widely with former students moving into teaching careers. produced a short volume
  • My former colleagues Pierre Schlag and David Skover, who produced a short volume early in their careers that catalogued the Tactics of Legal Reasoning (1985).
  • Richard Michael Fischl and Jeremy Paul, Getting to Maybe: How to Excel on Law School Exams (1999)
  • Leading clinical teachers Albert J. Moore and David Binder, Demystifying The First Year of Law School: A Guide to the 1L Experience (2009)

In recent decades much of the heavy lifting in legal reasoning has devolved upon teachers of legal analysis, research and writing. Among the results is a burgeoning literature proposing variations on the syllogistic Issue-Rule-Analysis (or Application)-Conclusion approach to analyzing and writing about legal problems, as well as a variety of textbooks.


Legal Education & Civility in the Legal Profession

A recurrent theme in current critiques of legal education is the need to develop lawyers with interpersonal, intrapersonal, and leadership knowledge, skills and values, as well as the traditional analytical skills and doctrinal knowledge. (A significant portion of Chapter 6, Teaching the Newly Essential Knowledge, Skills, and Values in a Changing World in the recent volume Building on Best Practices: Transforming Education in a Changing World (Lexis 2015) is devoted to the what and how of teaching such topics.)

Opportunities to reflect on this theme abounded in early October, when I had the privilege of attending the Civility Promise Continuing Legal Education seminar in Sovana, a small hill town in southern Tuscany, Italy. Sponsored by Seattle University Law School. and Robert’s Fund, the seminar brought together fifteen attorney participants from diverse practice backgrounds. They included a retired corporate attorney and managing partner of what is now a leading global law firm, a retired trial court judge, and lawyers with criminal or civil litigation, or transactional practices in both private and government settings.

Conceived by Paula Lustbader, teacher extraordinaire and emeritus professor of law at Seattle U. in collaboration with Italian artist Sergio Tamassia, the seminar was co-taught by two exceptionally skilled presenters: Tim Jaasko-Fisher, Senior Director of Curriculum and Programming for Robert’s Fund, formerly Assistant Attorney General and then Director of the University of Washington Law School Court Improvement Training Academy, and Craig Sims, Chief of the Criminal Division of the Seattle City Attorney’s Office.

The seminar identifies three pillars of civility: consciousness, community, and creativity. After fostering each pillar within the group in a brilliantly executed mix of didactic, reflective, and creativity-facilitating teaching methods, participants are challenged to take their learning into the profession.

Each participant was drawn to the seminar for their own personal reasons and several shared compelling experiences — the opposing counsel whose business model was the shake down, the ultimately unsuccessful malpractice suit based on the theory that an attorney approaching a case with a collaborative mindset violated her duty to her client, the former colleague who cracked under pressure and – the ultimate case of incivility — murdered his opposing counsel. And all bemoaned the all-too-common misconception that the adversary system is about behaving uncivilly, rather than developing and presenting the most compelling arguments on the merits.

Concerns over incivility have led some jurisdictions to adopt mandatory civility codes and help inspire the burgeoning mindfulness movement. Like the profession, many law schools are pursuing mindfulness for multiple reasons, including encouraging civility. Whether these efforts will be sufficient to effect widespread change in individual attorney behavior and the culture of the legal profession remains to be seen. But the Civility Promise seminar provided both incentive and tools for change. We can also hope that it will inspire similar efforts in legal education.

Providing Pro Bono Opportunities and Field Placements at Private Firms

Law schools working to provide students with substantial pro bono opportunities may look beyond traditional public interest firms, legal services providers, or governmental agencies. The idea of allowing students to assist in representing pro bono clients at private firms is appealing. What a great way to encourage and expand the pro bono efforts of the local bar, while helping students develop professionalism and an understanding of a lawyer’s responsibility to the community! Typically, the bench and bar love the idea. Such programs could provide much-needed legal services to persons of limited means, while reinforcing and modeling the importance of pro bono legal service.

But among the challenges to creating these programs at for-profit firms is the need to clearly distinguish such arrangements from an employment relationship. Similar issues arise when law schools explore the possibility of field placement programs at for-profit firms. Cautious directors of field placement and pro bono programs have been watching these issues develop, as the U.S. Department of Labor and the ABA have traded correspondence in an effort to provide clarity.

Recent federal court decisions have cast doubt on whether the DOL’s interpretation of the issue will be followed. In Schumann v. Collier Anesthesia, the Eleventh Circuit sided with a prior ruling by the Second Circuit, putting aside some of the concerns of the DOL.  As reported in the National Law Journal:

On September 11, 2015, the Eleventh Circuit became the first appellate court to address the standard for lawful unpaid internships since the Second Circuit’s ruling in Glatt et al. v. Fox Searchlight Pictures, Inc. et al. The new decision adopts the Glatt test and reasoning wholesale, and provides helpful guidance on applying the Glatt factors. The case also strengthens the trend away from relying on the DOL’s Fact Sheet 71, which purports to provide restrictive guidance on unpaid internships. ….

Echoing the Second Circuit in Glatt, the Eleventh Circuit disagreed with the SRNAs and rejected the DOL’s six-factor test. As a threshold matter, the court noted that because the six-factor test is not a regulation it is only due deference “proportional to its power to persuade.” The court “[did] not find it persuasive.”

Does this change anything? Now what?

We want to provide opportunities for students to participate in well-structured field placement programs and for them to engage in introductory pro bono work. Private firms are perhaps not the best place for this, but many law schools need to see whether somehow it can be made to work – serving the educational needs of the students while improving the provision of pro bono legal services.

The 25 Most Important Lawyering Skills?

In discussing bar exam reform in my earlier post, I referenced the results of this job analysis survey of newly licensed attorneys. The attorneys, all in practice for three years or less, were asked to rate the significance to their jobs of various skills or abilities (e.g., legal reasoning, organizational skills, written communication) and various knowledge domains (e.g., Rules of Evidence, Contract Law, Rules of Civil Procedure). Ever since I first saw the results, I have been taken with one particular statistic: The respondents rated 25 different skills or abilities as more significant to their jobs than the highest rated knowledge domain.

After the results came out, I looked more closely at these 25 skills and organized them into five broader skill categories. (My chart, which includes all 25 skills and each one’s average rating on a scale of 1 to 4, is below.) I then led a discussion on the importance of all of this to legal education at a legal writing conference last spring. Some of the colleagues in attendance offered insightful and practical comments that I’d like to share here.

One suggested that the 25 skills are a good starting point for formulating a new course to satisfy the ABA’s expanded practical skills requirement in the new Standard 303(a)(3). Others suggested that my chart, or something akin to it, could be a means for identifying and measuring learning outcomes for “other professional skills needed for competent and ethical participation as a member of the legal profession” under Standard 302(d), or additional learning outcomes under Interpretation 302-2.

I hope that many in legal education will find this chart, my colleagues’ ideas, and the overall survey results to be valuable tools. And, if anyone has feedback on how to revise the chart to make it a more useful tool, please get in touch.

Communication Analysis Research Project Management Professionalism
Written communication 3.77 Critical reading & comprehension 3.55 Computer skills 3.28 Paying attention to details 3.67 Professionalism 3.58
Listening 3.60 Synthesizing facts & law 3.55 Electronic researching 3.26 Using office technologies 3.56 Judgment 3.29
Oral communication 3.58 Legal reasoning 3.54 Fact gathering & evaluation 3.22 Knowing when to go back & ask ?s 3.46 Diligence 3.26
Interpersonal skills 3.44 Issue spotting 3.43 Organizational skills
Answering questions succinctly 3.30 Information integrating 3.10 Working within established time constraints 3.44  
Advocacy 3.24 Decisiveness 3.31
Consciousness of limitations 3.15
Planning & strategizing 3.13



Get every new post delivered to your Inbox.

Join 993 other followers

%d bloggers like this: