More on Law Schools and Access to Justice

Earlier in the week, Professor Margaret Moore Jackson had this post about law schools and access to justice.  This follow up post highlights some recent developments in this area and explains some of the ways in which law schools might approach embracing an access-to-justice mission.

A recently released report from the U.S. Legal Services Corporation (LSC) provides damning information on the state of the justice gap, which it appropriately calls the justice “gulf”.  The LSC’s main findings include the following:

  • 70% of low-income households have experienced at least one civil legal problem in the last year;
  • 70% of low-income Americans with civil legal problems reported that at least one of their problems affected them very much or severely;
  • Low-income Americans with legal problems seek assistance only 20% of the time, driven by the cost of services, not knowing that their problem is of a legal nature, or not knowing where to turn for help;
  • An estimated 1.7 million low-income Americans will turn to LSC-funded programs for legal help and yet more than half of these will receive only limited help or no help at all.

These figures should give the entire legal profession in general, and law schools in particular, great pause.  The access-to-justice crisis in the United States threatens to undermine our system of justice, as an adversarial system where, more and more, one side does not have representation, is not a system of justice at all.  Just as members of the legal profession have an ethical duty to ensure access to justice, so, too, do law schools have a responsibility to prepare their students to enter the profession with the mindset that we must close this justice gulf.

To these ends, on May 18, 2017, New York State’s Permanent Commission on Access to Justice held its sixth annual “Law School Access to Justice Conference” at St. John’s School of Law.  This robust conference manifests the Permanent Commission’s view that law schools have a critical role to play in closing the justice gap.  Court officials, law school faculty and administrators, and a large number of practitioners all gathered for this one-day conference to explore ways to work more closely together to help address the access-to-justice crisis plaguing the United States.  Such a conference is timely and worthwhile because law schools are in a terrific position to help close the justice gap as they educate their students to enter the profession, hopefully with an appreciation for the legal needs of all Americans and a sense that helping to close this justice gulf is the responsibility of all lawyers.

The legal profession, law schools that produce the lawyers who will join the profession, and individuals and families who would like the assistance of a lawyer all have shared interests: a desire to increase access to justice. Through this convergence of interests, these critical stakeholders can come together to advance meaningful social change that improves such access, strengthens civic engagement, and supports the rule of law. The focus of such efforts would be to develop a stronger appreciation for the role lawyers can play in society and the difference lawyers can make in furthering civic virtues.  Law schools can thus embrace an access-to-justice mission as a way of advancing not just their own interests, but those of the profession and society as a whole.

What would embracing an access-to-justice mission mean for law schools?  The cornerstones of law schools are the teaching, scholarship, and service that the faculty and staff at such institution undertake.  Should law schools embrace an access-to-justice mission, doing so would impact these three areas.  While an access-to-justice mission often permeates different divisions within a law school—like its clinical or pro bono programs—embracing an access-to-justice mission would mean working across the institution to close the justice gap, in the school’s teaching, scholarship, and service.

First, with scholarship, there is a growing body of research on the difference lawyers make in the lives of their clients.  A wonderful meta-study of recent research can be found here.  While not all of this research finds universal benefits from all legal representation, more such research, which tends to be interdisciplinary, is needed, and law school faculties should explore opportunities to work with scholars from other disciplines to assess the differences lawyers can make in the lives of their clients to help make the case for broader access to justice.  Similarly, scholars can build on such work to do cost-benefit analyses of providing government funding for free legal representation in different contexts.  Research consistently shows that providing funding to offer free lawyers to low-income clients saves local and state government millions of dollars (see example here).  Legal services lawyers help prevent eviction and homelessness, help individuals obtain stolen wages (which then make their way into local economies), and gain access to federal benefits.  By developing this research more, local and state governments can see the financial value of investing in free legal services, and legal scholars, teaming up, again, with other disciplines, can help make the case for expanding access to justice.

We can also incorporate access-to-justice issues in teaching and service as well.  While one might deal with access-to-justice questions in a class on legal ethics, the consequences of the justice gap for legal doctrine is profound in many areas of law, from constitutional law to consumer and criminal law.  We can incorporate access-to-justice questions across the curriculum quite easily, exploring, in a trans-substantive way, the extent to which the justice gap affects doctrine and practice in all areas of the law.

Similarly, service is another core component of law schools and all faculty and staff can be engaged in efforts to improve access to justice and raise the profile of access-to-justice questions by participating in pro bono efforts, talking about how students can engage in such efforts, and stressing the importance of incorporating public service into all of our students’ careers, regardless of whether they engage in public interest law per se or work in other sectors.  Since all lawyers have an ethical obligation to improve access to justice, students should learn, early on, the ways that access to justice can and should permeate everything lawyers do throughout their careers.

Law schools presently face considerable challenges, not the least of which is ensuring that their students have viable and rewarding careers once they leave the law school and embark on their professional lives.  By embracing an access-to-justice mission, law schools can possibly help make the case for expansion of funding for free legal services but also ensure that their graduates are committed to access-to-justice principles throughout their careers.   Doing so will not just help reduce the justice gap and law graduates’ career options and opportunities, it will help strengthen the role the profession and law schools play in ensuring our justice system works for everyone.

For a deeper exploration of many of these issues, see here; comments and feedback welcome.


I explore some of these issues in greater depth here; I welcome comments and feedback.

Recalibrating the Mission to Pursue Access to Justice

Raymond H. Brescia’s recent article (When Interests Converge: An Access-to-Justice Mission for Law Schools, 24 Geo. J. on Poverty L. & Pol’y 205 (2017)) identifies opportunity within a trio of connected challenges:  declining law school enrollments, decreased job prospects for graduates, and the ever-present legal services gap. Building upon Derrick Bell’s “Interest-Convergence Theory,” Professor Brescia posits that law schools should revise their missions to explicitly address the goal of enhancing access to justice, saying:

This interest convergence represents an opportunity for law schools to embrace social change that expands access to justice and restores the role of the legal profession in promoting democratic values, democratic institutions, social equality, and the rule of law.

Whether we regularly graduate a glut of new lawyers (as in the late 1980s/early 1990s) or see dramatic decreases in those numbers, millions of people still go without legal advice on their rights or representation in lawsuits. We need lawyers, and there is plenty of work. I agree that some might be more interested in pursing a law degree if they saw a clearer path toward contributing to access to justice. Schools with this mission would need to consider how to graduate new lawyers with a combination of legal knowledge/skills, a reasonable debt load, and a realistic business model – one that allows them to provide legal representation to individuals with average and lower incomes. If calibrated correctly, this re-envisioned mission might not only attract law school applicants, but also could redirect legal education in ways that improve the profession and address the ongoing need for lawyers.

What might be most striking about this proposal is how it could address more than than the sustainability of law schools, the employment of law grads, or the continuing lack of legal representation for ordinary individuals. Most of us care about all of the above. But the role of legal education and lawyers in supporting democracy and rule of law is on my mind a lot these days. Law schools that focus their missions more intently upon access to justice explicitly advance the values underpinning democracy itself, as Brescia notes in his article. David Leonhardt has written about present risks to democracy and the rule of law. Others view recent events as re-establishing the rule of law over a runaway administrative state. No matter how you see it, educating lawyers about their role in promoting an enduring democracy is a critical mission. As Steven Harper said, “The stakes are high and have nothing to do with politics or party loyalty.”

More on Challenges in Motivating Law Students to Learn

Thanks for your response, Robert.  I definitely agree that the Socratic method and other common teaching practices, such as use of casebooks, can reduce students’ motivation to learn (though some faculty use these methods successfully).  I certainly found them counter-productive when I was a law student.

I taught dispute resolution courses where the reading and class discussion emphasized the practical application of the material.  I didn’t use the Socratic method or assign students to read cases and I still had problems with the motivation of a substantial proportion of my students.  Part of the problem in my courses was that I didn’t call on students to recite cases and I usually didn’t have exams, so some students probably felt that they could “blow off” the reading assignments without jeopardizing their grades.  I generally relied on volunteers to speak in class, though I periodically called on students who didn’t contribute much.

I agree that students merely reading the assignments is not the same as their actual learning the material – but usually it is a pre-condition for learning.  Students who don’t do the reading assignments are less likely to learn as much as those who do the reading.

While the content of my quizzes didn’t fully capture what students needed to perform well in class and written assignments, I chose questions that focused on useful building blocks for what they needed to know.

Your suggestions for how to conduct quizzes make sense to me.  I can also imagine that faculty could design quizzes in other ways to both help motivate students and give formative feedback.

The larger point of my post was to identify problems of (lack of) student motivation to learn and to stimulate thinking about various techniques, not limited to quizzes, to elicit students motivation.  In the first instance, it’s students’ responsibility to be motivated and perform their obligations as students.  No one is holding a gun to their heads to attend law school.  That said, given the current realities of legal education, dealing with lack of motivation by some students is a fact of life for many faculty.

Challenges in Motivating Law Students to Learn

This post riffs on this post, which describes benefits of formative assessments and suggests some ways to do them.  It cites research showing that rewards aren’t sufficient motivation for students to learn and argues that using good formative assessments can increase students’ motivations to learn.  It suggests using low- or no-stakes formative assessments.  For example, at the end of a class, faculty could have “students answer three questions on an index card and turn it in: 1) One thing I learned in class today; 2) One thing I still have questions about; 3) Where will I use what I learned?”

I agree that formative assessments are pedagogically appropriate and can help motivate students to learn.

Unfortunately, due to the pernicious influence of competitive law school grading culture, among other things, a substantial proportion of students are motivated primarily to get good grades and not so much to learn.  As a result, such students do not regularly prepare for class, figuring that they can “get away” without doing the reading assignments.  Often, they are right.  In any given class, there is a good chance that they will not be called on to discuss the readings, and if they are called on, they may bluff or simply “pass.”

Unannounced, Short, Ungraded Quizzes to Increase Motivation and Feedback

To help motivate students to learn, I used a variation of the above approach, which combined incentives to do the reading assignments with low-stakes formative assessments.  I informed students in my syllabus and on the first day of class that there would be unannounced and ungraded pop quizzes on some days.  I told them that the quizzes were intended to help identify what they learned from the readings, prepare them to discuss issues in class, and motivate them to do the readings.

The quizzes consisted of one question about a major issue in the reading that they would be able to answer easily if they had done the reading.  Indeed, reading their answers, it was generally clear if they had read the assignments.  Students had to put their names on their quizzes, and this system enabled me to efficiently determine if which students had not done the reading without embarrassing them in public.  Although failing to give a good answer did not affect their grades, students generally were privately embarrassed if they hadn’t done the reading.

Invariably, a substantial minority of students did not do the reading before the first quiz and I would announce this fact in the following class.  As a result, increasing proportions of students were prepared for later quizzes, especially since they never knew in which classes I would give a quiz.  Based on prior experience, I was convinced that, without the quizzes, the proportion of students reading the assignments before class would have declined throughout the semester.

I generally gave students 1-2 minutes to complete the quizzes and, right after I collected them, we would discuss the question, which addressed a key part of the material for the day.  After class, I read the quizzes, which didn’t take much time.   At the beginning of the following class, I would comment about students’ responses, noting where students had misunderstandings.  Usually, I didn’t need to give students individual comments, though some answers were so problematic that I would send students emails about them.

Although the quizzes were not graded, students seemed motivated to give as good answers as they could.  I think that they were motivated not to seem dumb – and they had to write something anyway, so they might as well do a decent job.  The syllabus stated that although the quizzes generally were not graded, grades could be adjusted to reflect repeated unusually good answers or inability to answer appropriately.  I never had to adjust the grades.

I had various reactions from students about this system of quizzes.  Naturally, there was some grumbling.  Mostly, it was good-natured and when giving a quiz, I joked about how they would have a chance to do one of their favorite activities.  I suspect that having to do the quizzes didn’t make much of a difference for most students because the questions seemed reasonable – not nit-picky – and it wasn’t much of an imposition.  Some students said that they liked the quizzes I think because it motivated them to do the reading, which they might otherwise have skipped.

Part of my motivation for using this system was to improve the quality of class discussion.  Many faculty probably have had that horrible sinking feeling when students’ minds were not present in class even though their bodies were there.  When many students haven’t done the reading, it’s hard to have a good class discussion – and it discourages students from preparing.  If a substantial proportion of students regularly don’t prepare for class and “get away” with it, others may feel like suckers, wondering why they should prepare.

Increasing Students’ Motivation to Learn

Undoubtedly, there are many teachers who inspire students much better than I can. There certainly are many diligent law students who are intrinsically motivated to get the full advantage of their legal educations.  It is sad, however, to teach in an environment where a non-trivial proportion of soon-to-be-lawyers don’t fulfill their obligations to complete their work and, instead, try to game the system.  It would be nice if it wasn’t necessary to use techniques like my quizzes to motivate such students.

Of course, we must work within our systems as they are (or as they are changed).  In my courses, I think that the quizzes contributed to better class discussion and, hopefully, improved learning.  Much more change in the educational culture would be required to reduce the need for techniques like these.

Have you had problems motivating students to learn?  What techniques have you tried and how well did they work?  Are there any articles or resources you would suggest about handling these issues?  Please share your response in a comment.

Studying Better Ways to Test Bar Applicants for Minimum Competence: Another Reason to Care about the California Bar Exam (besides the cut score debate)

In addition to her post on Law School Café about alleged correlations between bar exam scores and lawyer discipline (discussed on this blog here), Professor Deborah Merritt recently offered another bar exam-related post. This one provides intriguing historical perspective on the current need to expand the range of skill sets tested on the bar exam. Following up on points made by Professor Derek Muller, Professor Merritt discusses a 1980 study by the California Committee of Bar Examiners, cosponsored by the National Conference of Bar Examiners (NCBE), on adding a clinical component to the bar exam. Several hundred applicants who had taken the July 1980 California Bar Exam volunteered to complete an additional clinical evaluation requiring them, among other things, to interview a client and examine witnesses. Professional actors played the role of clients, akin to the standard patient role that actors perform for clinical evaluations in medicine. The applicants were scored based on an elaborate protocol.

Delving into the statistical results of the study, including comparisons between outcomes on the conventional bar exam and outcomes on the clinical evaluation, Professor Merritt illuminates how crucial it is nearly 40 years later for bar examiners to study and implement alternative assessments of skills not currently evaluated by the bar exam. She points out that, while the study’s results were by no means definitive, they at least suggest “the disturbing conclusion that a significant percentage of conventional bar passers (about two of every five) lack basic practice skills that are essential in representing clients.”

I find this discussion particularly apt in 2017, the 20th anniversary of the first administration of the Multistate Performance Test (MPT), the written skills test now a part of the bar exam in 40 states and D.C.  What started the path toward written performance testing and the MPT? A study conducted by the California Committee of Bar Examiners (cosponsored by the NCBE), possibly the same one referenced by Professor Merritt.  On the occasion of the MPT’s 10-year anniversary in 2007, the Bar Examiner, a magazine published by the NCBE, briefly described the California-based origins of the performance test and indicated that the MPT was ultimately based largely on “the California model.” (The piece, in the November 2007 edition of the Bar Examiner, is apparently not retrievable online.)

Written performance testing was the last meaningful innovation in bar exam testing. In thinking about who might lead an effort toward the next one that introduces greater clinical evaluation, including possibly of oral skills, I think not of a top-down effort from the resolutely conservative NCBE. It is focused on getting as many jurisdictions as possible to adopt its Uniform Bar Exam (26 and counting as of today). Rather, I think of a bottom-up effort by individual states—perhaps with California in the lead—serving as laboratories for testing methods that could ultimately spread to other jurisdictions, thereby persuading or forcing the NCBE to join.

The history of written performance testing is illustrative of my point. Long before the NCBE went forward with the MPT in 1997, not just California but also Alaska and Colorado devised performance tests of their own and administered them on the bar exam. Indeed, those three states were administering performance tests in the early 1980s, playing an important initial role in advancing the cause of a needed bar exam reform. Here, for example, is a follow-up study of the 1983 California Bar Exam, discussing its two performance tests.

The biggest barrier to innovation at the state level is the NCBE’s influence, which increases with each state that adopts the UBE and thereby constrains itself to offer the conventional bar exam that the NCBE requires it to. Indeed, both Alaska and Colorado, two of the original performance test states from the 1980s, have adopted the UBE, meaning neither of those states will be doing any more bar exam innovation. That leaves California (and any of the other 23 states that have yet to join the UBE, none of which matches the influential profile of California).

Why the California study or studies did not lead to some form of clinical evaluation beyond written performance tests is unclear, though two obstacles that come to mind are expense and testing reliability. Indeed, the 1980 study that Professor Merritt references summarized one of its findings as follows: “[T]he relatively low reliability, administrative difficulties, and high costs associated with most (but not necessarily all) standardized oral tasks probably precludes even considering them as possible components of a general bar examination. Written tests of clinical skills, on the other hand, are relatively easy to construct, administer, and score. Further, unlike oral tasks, the score on written tasks are moderately correlated with one another.”

It seems worthwhile to revisit those conclusions, given the passage of time and possible advances in testing methods, and given that the medical profession requires clinical evaluation of its applicants.  Today, 24 years after the MacCrate Report, 20 years after the advent of the MPT, and 10 years after the Carnegie Foundation Report, the legal profession needs a better bar exam.  I join Professor Merritt’s call for a national task force on the bar exam, sponsored by AALS, the Conference of Chief Justices, the ABA Section of Legal Education and Admissions to the Bar, and maybe even the NCBE.  As Professor Merritt writes, such a task force could “study current approaches to the bar exam, develop a more realistic definition of minimum competence, and explore best practices for measuring that competence.”

But I also come back to the states, and to California specifically. There is a vigorous debate going on about whether California should lower its bar exam cut score. That’s an important discussion to have. But I might suggest another discussion to have about the California Bar Exam: Shouldn’t California resist the UBE and instead conduct a new study of alternative methods for assessing today’s relevant lawyering skills that are not encompassed by the UBE?

Intelligence: Stagnant or Malleable? Exploring Formative Assessment

Formative assessment isn’t just a new requirement handed down by the ABA, it is also a helpful tool to maximize students’ motivation and professors’ learning. Students who are highly motivated to learn, learn more and perform better when their knowledge is tested. But what motivates students to learn? Many believe that extrinsic rewards – like grades or prestigious jobs – motivate students to learn. But decades of research by Stanford professor, Carol Dweck, and other psychologists, has shown that extrinsic rewards are insufficient motivators. Instead, professors can foster more robust learning by introducing students to and reinforcing a growth mindset.

Dweck discovered that students’ implicit beliefs about intelligence drive their reactions to feedback, goal orientation, effort and persistence, profoundly affecting their learning. Students who believe that intelligence is a fixed trait, that it remains fairly stable over the course of one’s lifetime, behave in maladaptive ways when faced with academic difficulty or setbacks. These students are said to have a fixed mindset. On the other hand, students who believe that intelligence is malleable, that people can significantly increase their intelligence, welcome feedback that will help them learn. They also take on more challenging work and persist through difficult intellectual challenges. In study after study, students with a growth mindset out-perform students with a fixed mindset when they face difficult problems.

Although many students enter law school with a fixed mindset, students’ mindsets can change. Researchers change students’ mindsets by merely having them read a chapter on how intelligence can grow, just like a muscle, when students effortfully engage their brains. Teachers and coaches play a key role in shaping students’ beliefs about intelligence. Professors who reinforce the notion that every student can get much better at the skills they need to perform well in law school – and in the legal profession – can make a significant impact on students’ lifelong motivation to learn.

Formative assessments inform both the professor and the student about the student’s learning. Ongoing formative assessments should be low stakes, i.e., low or no point value, and high reward, i.e., accurate for demonstrating where a student’s skill level currently is and how he or she might improve. In other words, formative assessments help professors understand where students are in the learning process, how much more they need to learn, and how they can close that gap. As Dweck has said, formative assessment is about “telling the truth about current achievement and then, together, doing something about it, helping [the student] become smarter.”

Formative assessments help students develop a growth mindset and increase motivation because they focus on the learning process rather than the result. Formative assessment also educates professors, often requiring them to teach more explicitly the skills they are assessing and provide students with targeted feedback and strategies designed to help them master those skills. It gives students an honest evaluation tied to specific learning outcomes for a deeper understanding or a better performance. It also forces professors to grapple with the material they are teaching in a more concrete and systematic way.

Try some low stakes formative assessments in your next class. For example, at the end of a class, have the students answer three questions on an index card and turn it in: 1) One thing I learned in class today; 2) One thing I still have questions about; 3) Where will I use what I learned? This will give you an idea of how you can help students better understand the material and how it fits into their long-term goals. In our clinic, we often use art as a formative assessment. For example, when we teach students the procedural rules they will need to know when reviewing their clients’ files, we provide students with the pretrial procedure statutes and have them draw the process as they understand it. They share their drawings with the group, and we critique them. Do the drawings accurately reflect the rules? Do they leave anything out? Why does it matter?

Essentially, formative assessment is meant to educate the student and the professor about what the student currently understands and how the professor can help improve student learning. It also creates an ideal opportunity to increase student motivation by reinforcing a growth mindset – that students will get smarter and more skilled if they put forth effort with the right strategies to increase learning.


Professor Merritt’s Blog post on attorney discipline and bar exam WORTH A READ!

Our blog has often posted about many issues related to licensing lawyers, experiential requirements for admission, the monopolizing power of the NCBE and the pros and cons of the UBE.  Thus, I recommend to our readers an excellent post by our blogger friend Professor Deborah Merritt over at Law School Cafe on bar exam scores and lawyer discipline. Professor Merritt analyzes an article by Pepperdine Professors Robert Anderson and Professor Derek Mueller entitled The High Cost of Lowering the Bar Exam.   Professors Anderson and Mueller opine that “lowering the bar examination passing score will likely increase the amount of malpractice, misconduct, and discipline among California lawyers.” Merritt objects to any causal inference noting,

Two key facts, however, weigh strongly against drawing that type of causal inference. First, as Anderson and Muller point out, “[t]here is virtually no discipline in the first 10 years of practice.” If the bar exam measured qualities related to attorney discipline, one would expect to see disciplinary cases emerge during those 10 years. Wouldn’t attorneys with marginal competency (as measured by the current bar exam) reveal their deficiencies during their early practice years?

Second, attorney discipline almost never rests on lack of knowledge about legal doctrine, poor reasoning skills, or bad writing–the skills currently measured by the bar exam. Levin and her colleagues reported that attorneys most often received discipline for failing to communicate with clients (20.0%), lack of diligence (17.93%), and failure to safeguard client property (11.26%). Only 4.14% of disciplinary sanctions related to “competence”–and even some of those cases may have reflected incompetence in areas that are not tested by the bar exam.

My favorite comment by Professor Merritt provides another example from which we should not infer causality (however tempting it might be to some of us who have been hurt by patriarchy),

We should not exclude individuals from a profession based on qualities that merely correlate with misconduct.

To underscore that point, consider this: The strongest predictor of attorney discipline is the y chromosome. Male attorneys are much more likely than female ones to be disciplined. If we want to use correlations to reduce instances of attorney discipline, it would be much more efficient to ban men from the profession, subject them to special character exams, or require them to achieve a higher bar exam score than women. Those actions, of course, would raise special issues of gender discrimination–but they illustrate the drawbacks of predicting malfeasance based on correlations.

These questions and assumed correlations are important ones. Many defend the decreasing bar passage statistics as appropriate market correction to prevent “undesirables” from entry into the profession — a consumer protection argument. However, as Professor Merritt points out, there is so much more to unpack here. For example, most misconduct challenges occur against solo practitioners or small firms. This raises overlapping socio-economic questions: which lawyers could be perceived as easiest to challenge, which lawyers have the best legal defense teams, and which kind of clients have the most reason to complain.

After teaching for over 28 years and observing which graduates pass the bar on the first try and which do not , I am skeptical of the Anderson-Mueller argument. I would love to see the NCBE and other scholars engage in a socio-economic analysis of bar passage and of disciplinary misconduct.

%d bloggers like this: