Unmasking Assumptions about Employment Outcomes and Legal Education

In an upcoming Wisconsin Law Review article, Robert Kuehn, Associate Dean for Clinical Education and Professor of Law at the Washington University Law School, presents a cogent, well-supported and thoughtful article describing the limitations of and lessons we can learn from the existing empirical analysis correlating student enrollment in clinical education and employment outcomes.  Kuehn’s article, entitled Measuring Legal Education’s Employment Outcomes is particularly powerful because it provides a thorough empirical rejection of the claim that clinical coursework might actually harm employment outcomes, as asserted by Professor Jason Yackee and which attracted some sound-bite attention earlier this year. In what is, perhaps,  an unexpected twist, Kuehn demonstrates that using Yackee’s statistical assumptions and methodology also would produce negative correlations for those students who participate on law journals or in moot court competitions.  Kuehn argues that one can’t draw any reliable conclusion from Yackee’s 2013 model, and perhaps not from any nationwide statistical model – as opposed to a particularized analysis of one school –  on the likely effect of clinical courses (or other activities like law journal or moot court) on employment, and surely not the negative effect Yackee posits. Kuehn points out that as to clinical coursework, the available evidence (through surveys) indicates that such experiences do aid some students in securing employment.

If you, like me, still become a bit nervous about how much you actually remember from undergraduate statistics courses, do not be alarmed by this post!  You will find Kuehn’s article accessible and a quick good read, even when he is using words like “regression analysis,” “granular data” and “variable choices.”   Here are the points made in Measuring Legal Education’s Employment Outcomes which I found most helpful:

  1. Kuehn’s reminder that when one confuses correlationwith causation one is bound to come up with a “misdiagnosis.” One problem with Yackee’s analysis is the lack of granular data to calculate the true employment rate for those who took a clinic (or who did not).  In fact, the data is so poor that “the results never account for more than half of the variability in employment across schools.”
  2. Kuehn’s explanation of the “confounding effect of prestige” and bar passage on employment outcomes.
  3. The problems of validity and reliability raised by analyses which employ information from ABA questionnaires, particularly those self-reports submitted prior to 2014.
  4. The fact that “13% of law schools” provide 80% of the school-funded jobs to law graduates. Not surprisingly, Kuehn found this factor biases many results if you examine nationwide statistics. And when Kuehn removes those jobs from the statistical analysis, Yackee’s correlation with clinical education falls apart even using his own assumptions and methodology.
  5. Yackee’s model yields completely different results if one uses the US News Lawyers/judges data versus academic peer data to control for the possible influence of perceived prestige.
  6. Application of Yackee’s model to “Law Journals” and “Skills Competition” and S. Newssub-groups also show no relationship to employment outcomes!
  7. In Yackee’s model, a better ranking is “strongly associated with improved employment outcomes.” However, Kuehn points out that a “closer examination of the relationship between rank and employment indicates that this positive association, although statistically significant when applied across the entire range of top 100 schools, does not hold true for schools ranked 51 through 100 (emphasis added).” 
  8. Kuehn’s documentation of employers who require, “strongly prefer” or identify law clinic experience as a positive factor in hiring such as The U.S. Department of Homeland, legal services and  legal aid offices, district attorney, public defender, fellowships and private law firms.
  9. Kuehn’s description of National Association of Law Placement (NALP) existing information: such as the  2011 survey of lawyers with non-profit and government offices;  the NALP survey of lawyers in firms of predominantly more than 100 attorneys; the NALP survey of public interest legal employers;  and the NALP 2013 presentation on the employment market reporting that ” law firms say they want new graduates to have ‘more experiential learning, client-based and simulation.”
  10. Kuehn provision of good information on other employer information such as the Lexis-Nexis WHITE PAPER: HIRING PARTNERS REVEAL NEW ATTORNEY READINESS FOR REAL WORLD PRACTICEProfessor Neil Hamilton’s employer survey to determine the relative importance of twenty-one different competencies in employer hiring decisions, and Professor Susan Wawrose’s legal employer focus groups which found employers prefer new hires with ” well developed professional or ‘soft skills” along with “strong fundamental practice skills.”

Professor Kuehn concludes by recommending that studies could best be done on a school-by-school basis by “surveying likely employers to find out what educational experiences of students are most valued.”  Professor Kuehn also recommends that schools could also “retrospectively look at various employment outcomes for graduates and any relationship” to students’ experiences while in school.

I agree with Professor Kuehn and am happy to report that  Albany Law School,  through its faculty Assessment committee and Admissions office,  is currently engaged in conducting employer focus groups and analyzing what best helps our students obtain employment in their desired career paths.  Until good data and information suggests otherwise, Professor Neil  Hamilton’s advice to law students,which Professor Kuehn quotes in his “must read” article, bears repeating:

In this challenging market for employment, a law student can differentiate herself from other graduates by demonstrating to legal employers that the student both understands the core competencies that legal employers and clients want and is implementing a plan to develop these competencies, including an ability to demonstrate that the student has experience with these competencies.

Annual Leadership in Legal Education Issue of Univ. of Toledo Law Review Filled with Best Practices Nuggets

The new issue of the University Toledo Law Review is out, featuring its annual “virtual symposium” on legal education by law school deans. These annual issues should be read not just be deans and people who are thinking about pursuing a law school deanship, but they should be read by college and university presidents and provosts, members of law school boards of trustees and advisory boards, senior administrative staff, and most important, by law school faculty. The articles in each volume, taken together, offer terrific insights into current challenges facing legal education, interesting historical background on various aspects of legal education, and innovative ideas to shape the future of law schools and legal education. The winter 2015 volume is no exception.

While I will not address all twelve of the articles/essays in this brief review, I do want to highlight several important themes in four pieces. Beginning with the opening contribution by two-time former dean Peter C. Alexander (Indiana Tech and Southern Illinois), more than mere references to “best practices” principles abound. One of Alexander’s assertions is that law schools, in “the new normal” must do more to create “practice ready” graduates as part of the ongoing curricular reform taking place. He also suggests, “Faculty members have to design new methods of instruction and create new pathways for students to learn….Deans must make funds available for faculty members to learn how people learn and how to teach the current generation of students.” (p. 263) This is an astute observation and one not lost on many in the academy. Most of us on the law faculty did not receive any formal education or degree in pedagogy. While those who work with students from pre-K through 12th grade must be certified as teachers after formal baccalaureate and post-baccalaureate training, there are no such requirements in higher education. Few, if any, dispute that in law school the learning styles of our students has changed over time, and this challenges law faculty to more attune to the need to change our teaching methodologies.

Another piece written by Professor George Critchlow, former interim dean and former director of the clinical programs at Gonzaga University School of Law, focuses on ensuring that legal education in a broad sense is accessible to those who wish to serve the public good – including non-lawyers (a good and controversial read). In his discussion on affordability, Critchlow reviews a number of ideas that have been circulating for years including, but not limited to: law schools partnering with legal services organizations and firms (resembling aspects of the medical school model); a discretionary third year program that consists entirely of a practice-oriented experience; participation by law schools with apprenticeship programs that allow or encourage students to engage in actual work outside of the law school in addition to classes (this goes well beyond the current law school supervised externship and clinic experiences); and cost savings to clinical programs by entering into “hybrid” arrangements with community based legal service providers.

A theme in Critchlow’s article is picked up in greater detail in an article by IIT Chicago-Kent College of Law dean Harold J. Krent and director of clinical legal education Gary S. Laser. Krent and Laser focus on meeting the experiential challenge through the operation of a fee-generating law clinic. By highlighting the example of the IIT Chicago-Kent model which in essence is organized as an in-house law office, the authors point out that students are exposed not just to the traditional live client experience of a clinic, but they develop an appreciation for the economics of law practice. This is important given the increasing attention that many law schools are giving to the business aspects of running law offices, whether it be through the incubator movement, the addition of courses on law office management, and the introduction of business skills to the curriculum.

The symposium ends with an essay by UC Hastings College of Law Dean Frank Wu which I highly recommend everyone read. Dean Wu offers his prescription for reforming law schools, much of which I will not address here due to space and my focus on best practice. Wu states, “A lawyer should be like a doctor. There is no medical school graduate who altogether lacks clinical experience. Every licensed physician has seen a live patient presenting actual symptoms before charging anyone for a diagnosis. Yet some law school graduates manage to do quite well by book learning alone. They need not interview, counsel, or draft, to earn honors, if their exams and seminar papers are good enough.” (p. 420) He discusses the increasing importance of the need for the academy and the profession to understand and appreciate the impact that technology is having and will have on the future of the practice of law and lawmaking. Wu addresses the ongoing and long-time debate over the profile of law professors as practitioners or intellectuals. (p. 440) In addressing the costs of change, Dean Wu asserts that the most expensive and most worthwhile change we have “recently” made in legal education is clinical legal education.

Every year I find fascinating the articles and essays published by the Toledo Law Review in their special “deans” issue. I am surprised that many people do not know that this annual symposium exists. It is a good read that should not be missed.

What’s going on in California? “TFARR- recommended” 15 credits of competency training

For those who did not closely follow the California State Bar debate on the requirement of 15 credits of competency training for bar admission (the work of the Task Force on Admissions Regulation Reform, or “TFARR”), I summarize the current status.  (Although I am currently co-prez of the Clinical Legal Education Association, known as CLEA, this post is not written with that hat on.)  This is my own thinking, albeit, informed by the excellent work of the CLEA Advocacy committee.

The TFARR process was two-staged, over a three year period, with opportunities for public comment throughout. CLEA  participated in that process and submitted five separate comments on the proposals that are available at http://www.cleaweb.org/advocacy under “Briefs and Other Advocacy” (documents 4-8).

In the end, TFARR recommended 15 credits of competency training which can be achieved in a variety of ways (in addition to how experiential credits can be earned under the new ABA regulations), and which include six credits of summer work. You can read the TFARR Phase II Final Report  at: http://www.calbar.ca.gov/AboutUs/PublicComment/Archives/2014PublicComment/201411.aspx

The process was complete in November, 2014, with final TFARR recommendations to the State Bar Board of Trustees (that responded to public comments) and unanimous adoption by the Board: http://board.calbar.ca.gov/Agenda.aspx?id=10891&tid=0&show=100008800&s=true#10013881 (agenda item 113). The TFARR Phase II FInal Report represents a compromise based on extensive input.

Lately, some confusion has arisen because of a letter posted to the AALS website authored by a non-standing committee of Deans.  The confusion arises because:

  1. Neither AALS nor this special Dean’s committee ever participated in the two stage TFARR process and so appear to be sort of “johnny come latelys, ” and
  2. The letter mistakenly focuses on an earlier draft of the final proposal failing to recognize the compromises already reached in the final proposal.

I understand that there are efforts underway to correct the confusion which makes me happy since the Deans’ letter is signed by two people whom I have long admired in a variety of contexts.

Other blogs are already exploring the 15 credit  proposal and its interesting and creative approach. For example,   “Kudos to California”  What do our readers think?

Building on Best Practices now available as eBook

Are you trying to:

  • Develop a meaningful law school mission statement?
  • Understand new accreditation requirements, learning goals, and outcomes assessment?
  •  Expand your experiential offerings?  Decide whether to use modules or courses?  An on-site clinic, an externship, or community partnership?
  •  Teach ALL of your students in the most effective ways, using a full range of teaching methods?
  • Add to your curriculum more of the professional identity, leadership, intercultural, inter-professional and other knowledge, skills, and values sought by 21st century legal employers?
  • Lead thoughtfully in the face of the challenges facing legal education today?

These and other topics are addressed in Building on Best Practices:  Transforming Legal Education in a Changing World,  now available in ebook format from LexisNexis at no charge.

The print version is not yet out.  LEXIS-NEXIS is taking advance orders for $50, plus shipping.  BUT we understand that they will make one copy available to every US legal educator for free upon on request.  Details on this and international availability still to come.

Thanks, and congratulations, to book project sponsor Clinical Legal Education Association (CLEA), the more than fifty legal educators who participated as authors, and the countless others who assisted as readers and in numerous other ways.

And, a huge shout-out to my wonderful and talented co-editors, Lisa Radke Bliss, Carrie Wilkes Kaas, and Antoinette Sedillo Lopez.

Inner Development, Community, Social Justice (Concurrent Session, AALS Conference on Clinical Legal Education)

Last, but not least, in this series highlighting lessons from experts in other disciplines relevant to how to navigate the chaotic “new normal”  in legal education: Thursday’s concurrent session organized by Tennessee’s Paulette Williams:  “A Commitment to Inner Development: Connecting the “New Normal” with Clinics’ Social Justice Mission”.

The session brought  Edward Groody and Timothy Dempsey from the Community Building Institute in Tennessee.  The Institute helps social service and criminal justice organizations become more effective by training participants in community building practices.  Taking an evidence-based approach built on motivational interviewing, trauma-informed care, and pro-social supports, community building is a “highly experiential process that helps participants remove barriers to communication and unlearn unproductive attitudes and behaviors.”

Groody began the session with a detailed overview of a four-stage process for building community:

  • Pseudo-community
  • Chaos
  • Emptying/Letting Go
  • Community

That process adds an important step — emptying/letting go — to Bruce Tuckman’s familiar “forming, storming, norming, performing” model of group formation.  My own interpretation of this additional,  third step is that it provides space for  participants to recognize,  and learn skills to address, the emotional issues that so often get in the way of honest connection with others.

Dempsey then shared powerful stories of how that process helps ex-offenders with post-prison re-entry,  allowing them to move past behavioral responses that may have seemed — and perhaps were — functional in their previous lives, but would block their efforts to move forward.   Or, to put it another way, this step acknowledges that in order to take advantage of education or employment opportunities, people need to let go of fears, resentments or trauma.  This is challenging work that is the foundation of many spiritual traditions, but can help build strong connections with others.

Time constraints prevented Paulette Williams from speaking in detail about how she makes use of this process in her clinical teaching work.  I hope she finds other forums for sharing those experiences and insights.

The insights of this community building process struck me as relevant not only to social justice and clinical legal education work, but also to faculty interactions within our law schools.  From another time and place, I well remember a year when every faculty meeting resulted in controversy, usually about something relatively minor that seemed to be a proxy for other, larger, but unacknowledged issues festering beneath the surface.    I suspect that many faculties are experiencing something similar as they operate  in the  current climate of uncertainty and change, too often getting stuck in the fear those conditions foster.  It’s  difficult for me to imagine applying this model in the typical law school environment.  But successfully moving through the “emptying/letting go” phase, as individuals and a group,  could be oh, so helpful!

Birth, Maturity, Creative Destruction & Renewal At AALS Clinical Conference

As someone who collaborated on a concurrent session titled “Facing Our Fears in Changing Times” at the AALS Conference on Clinical Legal Education, it’s probably not surprising that I was especially drawn to sessions that brought in models or speakers from other disciplines to provide insight on how to operate effectively in the midst of the current period of change in legal education.

In addition to my last post on Michele Weise’s Closing Plenary, in this and my next two posts, I’ll discuss three other provocative sessions that addressed different aspects of this theme.

On Tuesday morning my University of Washington colleagues Jennifer Fan and Lisa Kelly, worked with Rutgers-Newark’s Randi Mandelbaum and Syracuse’s Mary Helen McNeal to introduce the “liberating structures eco-systems model” of leadership.  That model views organizational change as an  infinity loop in which organizations move through four cycles that call for different styles of leadership:

Stage                                                   Leadership Style

Birth                                                     Entrepreneur

Maturity                                                Manager

Creative Destruction                           Heretic

Renewal                                               Networker

The model suggests that embedded in the cycle are two “traps“:

1. Between the Maturity and Creative Destruction stages lies the Rigidity Trap of “not letting go” of what the organization has birthed and brought to maturity.  Staying stuck in the past and wedded to the old ways of doing things.

2. Between Creative Destruction and Renewal lies the Poverty Trap of “not investing enough to accomplish renewal”.

Sound familiar? The session included an exercise where attendees decided which stage  they perceived their individual clinic, program, institution, or the clinical legal education movement to be in.  Participants  then added on the infinity loop diagram post-its with their results.  Although responses were spread around the loop, most clustered  among Maturity — Creative Destruction — and Renewal.  Most responses addressed clinical programs and law schools.

I find this framework a helpful reminder that our current struggles are “normal” and that they won’t last forever.  And inspiration to let go of fears and rigidity.

I’m grateful to my former colleague Tim Jaasko-Fisher for his work with liberating structures in the Court Improvement Academy of UW Law’s Children and Youth Advocacy Clinic.

Building on Best Practices for Legal Education Manuscript Submitted to Publisher

Four editors,  59 authors, 92 readers, three copy editors, librarians from two schools, a secretary, miscellaneous consultants, three student assistants for bluebooking, and one for setting up perrmacc links.*

Many people, occasionally in multiple roles, were needed to produce the manuscript sent to Lexis last Monday for the forthcoming book Deborah Maranville, Lisa Radtke Bliss, Carolyn Wilkes Kaas, and Antoinette Sedillo López (eds.),  Building on Best Practices:  Transforming Legal Education in a Changing World. (Lexis 2015).  A monster project — but, as I assured a friend, no, not a manuscript about monsters and not monstrously unpleasant to produce – just big, ambitious, and sometimes exhausting for the editors and authors.  A big thank you to all who participated!

The book is a follow up to CLEA’s Best Practices for Legal Education, the 2007 volume by Roy Stuckey and others that inspired this blog.  Like Best Practices, this book will be distributed for free to legal educators.  Lexis has promised to make it available in electronic format through their e-book library and to provide print copies on request.  Look for it in four to six months — if all goes smoothly perhaps in time for the AALS Clinical Legal Education Conference in early May.

The coverage of Building on Best Practices is wide-ranging.  To quote from the Introduction, “[t]his volume builds on the call to link mission and outcomes; emphasizing the themes of integrating theory, doctrine and practice, developing the broader spectrum of skills needed by lawyers in the twenty-first century, and taking up the question how best to shift law school cultures to facilitate change.”

Advance praise for the book has included:

  • “[M]ilestone in legal education . . . that legal educators will rely on as much as . . . on the first Best Practices book.”  (Patty Roberts, William & Mary)
  • “Educational for folks who don’t know much about experiential education and insightful for those who do. . . .Really something to be proud of . . . an invaluable resource to schools as they go to work on implementing the ABA’s new requirements for learning outcomes and assessment. . .The perfect product coming out at the perfect time.” (Kate Kruse, Hamline)

Once again, CLEA deserves kudos for its support of an important scholarly project on legal education.  And the Georgia State University, University of New Mexico, Quinnipiac University, and University of Washington Law Schools deserve a big round of thanks for supporting the co-editors in this project.

https://perma.cc/ provides an archive for those annoying website links that quickly become outdated.


Get every new post delivered to your Inbox.

Join 896 other followers

%d bloggers like this: