Unmasking Assumptions about Employment Outcomes and Legal Education

In an upcoming Wisconsin Law Review article, Robert Kuehn, Associate Dean for Clinical Education and Professor of Law at the Washington University Law School, presents a cogent, well-supported and thoughtful article describing the limitations of and lessons we can learn from the existing empirical analysis correlating student enrollment in clinical education and employment outcomes.  Kuehn’s article, entitled Measuring Legal Education’s Employment Outcomes is particularly powerful because it provides a thorough empirical rejection of the claim that clinical coursework might actually harm employment outcomes, as asserted by Professor Jason Yackee and which attracted some sound-bite attention earlier this year. In what is, perhaps,  an unexpected twist, Kuehn demonstrates that using Yackee’s statistical assumptions and methodology also would produce negative correlations for those students who participate on law journals or in moot court competitions.  Kuehn argues that one can’t draw any reliable conclusion from Yackee’s 2013 model, and perhaps not from any nationwide statistical model – as opposed to a particularized analysis of one school –  on the likely effect of clinical courses (or other activities like law journal or moot court) on employment, and surely not the negative effect Yackee posits. Kuehn points out that as to clinical coursework, the available evidence (through surveys) indicates that such experiences do aid some students in securing employment.

If you, like me, still become a bit nervous about how much you actually remember from undergraduate statistics courses, do not be alarmed by this post!  You will find Kuehn’s article accessible and a quick good read, even when he is using words like “regression analysis,” “granular data” and “variable choices.”   Here are the points made in Measuring Legal Education’s Employment Outcomes which I found most helpful:

  1. Kuehn’s reminder that when one confuses correlationwith causation one is bound to come up with a “misdiagnosis.” One problem with Yackee’s analysis is the lack of granular data to calculate the true employment rate for those who took a clinic (or who did not).  In fact, the data is so poor that “the results never account for more than half of the variability in employment across schools.”
  2. Kuehn’s explanation of the “confounding effect of prestige” and bar passage on employment outcomes.
  3. The problems of validity and reliability raised by analyses which employ information from ABA questionnaires, particularly those self-reports submitted prior to 2014.
  4. The fact that “13% of law schools” provide 80% of the school-funded jobs to law graduates. Not surprisingly, Kuehn found this factor biases many results if you examine nationwide statistics. And when Kuehn removes those jobs from the statistical analysis, Yackee’s correlation with clinical education falls apart even using his own assumptions and methodology.
  5. Yackee’s model yields completely different results if one uses the US News Lawyers/judges data versus academic peer data to control for the possible influence of perceived prestige.
  6. Application of Yackee’s model to “Law Journals” and “Skills Competition” and S. Newssub-groups also show no relationship to employment outcomes!
  7. In Yackee’s model, a better ranking is “strongly associated with improved employment outcomes.” However, Kuehn points out that a “closer examination of the relationship between rank and employment indicates that this positive association, although statistically significant when applied across the entire range of top 100 schools, does not hold true for schools ranked 51 through 100 (emphasis added).” 
  8. Kuehn’s documentation of employers who require, “strongly prefer” or identify law clinic experience as a positive factor in hiring such as The U.S. Department of Homeland, legal services and  legal aid offices, district attorney, public defender, fellowships and private law firms.
  9. Kuehn’s description of National Association of Law Placement (NALP) existing information: such as the  2011 survey of lawyers with non-profit and government offices;  the NALP survey of lawyers in firms of predominantly more than 100 attorneys; the NALP survey of public interest legal employers;  and the NALP 2013 presentation on the employment market reporting that ” law firms say they want new graduates to have ‘more experiential learning, client-based and simulation.”
  10. Kuehn provision of good information on other employer information such as the Lexis-Nexis WHITE PAPER: HIRING PARTNERS REVEAL NEW ATTORNEY READINESS FOR REAL WORLD PRACTICEProfessor Neil Hamilton’s employer survey to determine the relative importance of twenty-one different competencies in employer hiring decisions, and Professor Susan Wawrose’s legal employer focus groups which found employers prefer new hires with ” well developed professional or ‘soft skills” along with “strong fundamental practice skills.”

Professor Kuehn concludes by recommending that studies could best be done on a school-by-school basis by “surveying likely employers to find out what educational experiences of students are most valued.”  Professor Kuehn also recommends that schools could also “retrospectively look at various employment outcomes for graduates and any relationship” to students’ experiences while in school.

I agree with Professor Kuehn and am happy to report that  Albany Law School,  through its faculty Assessment committee and Admissions office,  is currently engaged in conducting employer focus groups and analyzing what best helps our students obtain employment in their desired career paths.  Until good data and information suggests otherwise, Professor Neil  Hamilton’s advice to law students,which Professor Kuehn quotes in his “must read” article, bears repeating:

In this challenging market for employment, a law student can differentiate herself from other graduates by demonstrating to legal employers that the student both understands the core competencies that legal employers and clients want and is implementing a plan to develop these competencies, including an ability to demonstrate that the student has experience with these competencies.

Annual Leadership in Legal Education Issue of Univ. of Toledo Law Review Filled with Best Practices Nuggets

The new issue of the University Toledo Law Review is out, featuring its annual “virtual symposium” on legal education by law school deans. These annual issues should be read not just be deans and people who are thinking about pursuing a law school deanship, but they should be read by college and university presidents and provosts, members of law school boards of trustees and advisory boards, senior administrative staff, and most important, by law school faculty. The articles in each volume, taken together, offer terrific insights into current challenges facing legal education, interesting historical background on various aspects of legal education, and innovative ideas to shape the future of law schools and legal education. The winter 2015 volume is no exception.

While I will not address all twelve of the articles/essays in this brief review, I do want to highlight several important themes in four pieces. Beginning with the opening contribution by two-time former dean Peter C. Alexander (Indiana Tech and Southern Illinois), more than mere references to “best practices” principles abound. One of Alexander’s assertions is that law schools, in “the new normal” must do more to create “practice ready” graduates as part of the ongoing curricular reform taking place. He also suggests, “Faculty members have to design new methods of instruction and create new pathways for students to learn….Deans must make funds available for faculty members to learn how people learn and how to teach the current generation of students.” (p. 263) This is an astute observation and one not lost on many in the academy. Most of us on the law faculty did not receive any formal education or degree in pedagogy. While those who work with students from pre-K through 12th grade must be certified as teachers after formal baccalaureate and post-baccalaureate training, there are no such requirements in higher education. Few, if any, dispute that in law school the learning styles of our students has changed over time, and this challenges law faculty to more attune to the need to change our teaching methodologies.

Another piece written by Professor George Critchlow, former interim dean and former director of the clinical programs at Gonzaga University School of Law, focuses on ensuring that legal education in a broad sense is accessible to those who wish to serve the public good – including non-lawyers (a good and controversial read). In his discussion on affordability, Critchlow reviews a number of ideas that have been circulating for years including, but not limited to: law schools partnering with legal services organizations and firms (resembling aspects of the medical school model); a discretionary third year program that consists entirely of a practice-oriented experience; participation by law schools with apprenticeship programs that allow or encourage students to engage in actual work outside of the law school in addition to classes (this goes well beyond the current law school supervised externship and clinic experiences); and cost savings to clinical programs by entering into “hybrid” arrangements with community based legal service providers.

A theme in Critchlow’s article is picked up in greater detail in an article by IIT Chicago-Kent College of Law dean Harold J. Krent and director of clinical legal education Gary S. Laser. Krent and Laser focus on meeting the experiential challenge through the operation of a fee-generating law clinic. By highlighting the example of the IIT Chicago-Kent model which in essence is organized as an in-house law office, the authors point out that students are exposed not just to the traditional live client experience of a clinic, but they develop an appreciation for the economics of law practice. This is important given the increasing attention that many law schools are giving to the business aspects of running law offices, whether it be through the incubator movement, the addition of courses on law office management, and the introduction of business skills to the curriculum.

The symposium ends with an essay by UC Hastings College of Law Dean Frank Wu which I highly recommend everyone read. Dean Wu offers his prescription for reforming law schools, much of which I will not address here due to space and my focus on best practice. Wu states, “A lawyer should be like a doctor. There is no medical school graduate who altogether lacks clinical experience. Every licensed physician has seen a live patient presenting actual symptoms before charging anyone for a diagnosis. Yet some law school graduates manage to do quite well by book learning alone. They need not interview, counsel, or draft, to earn honors, if their exams and seminar papers are good enough.” (p. 420) He discusses the increasing importance of the need for the academy and the profession to understand and appreciate the impact that technology is having and will have on the future of the practice of law and lawmaking. Wu addresses the ongoing and long-time debate over the profile of law professors as practitioners or intellectuals. (p. 440) In addressing the costs of change, Dean Wu asserts that the most expensive and most worthwhile change we have “recently” made in legal education is clinical legal education.

Every year I find fascinating the articles and essays published by the Toledo Law Review in their special “deans” issue. I am surprised that many people do not know that this annual symposium exists. It is a good read that should not be missed.

What’s going on in California? “TFARR- recommended” 15 credits of competency training

For those who did not closely follow the California State Bar debate on the requirement of 15 credits of competency training for bar admission (the work of the Task Force on Admissions Regulation Reform, or “TFARR”), I summarize the current status.  (Although I am currently co-prez of the Clinical Legal Education Association, known as CLEA, this post is not written with that hat on.)  This is my own thinking, albeit, informed by the excellent work of the CLEA Advocacy committee.

The TFARR process was two-staged, over a three year period, with opportunities for public comment throughout. CLEA  participated in that process and submitted five separate comments on the proposals that are available at http://www.cleaweb.org/advocacy under “Briefs and Other Advocacy” (documents 4-8).

In the end, TFARR recommended 15 credits of competency training which can be achieved in a variety of ways (in addition to how experiential credits can be earned under the new ABA regulations), and which include six credits of summer work. You can read the TFARR Phase II Final Report  at: http://www.calbar.ca.gov/AboutUs/PublicComment/Archives/2014PublicComment/201411.aspx

The process was complete in November, 2014, with final TFARR recommendations to the State Bar Board of Trustees (that responded to public comments) and unanimous adoption by the Board: http://board.calbar.ca.gov/Agenda.aspx?id=10891&tid=0&show=100008800&s=true#10013881 (agenda item 113). The TFARR Phase II FInal Report represents a compromise based on extensive input.

Lately, some confusion has arisen because of a letter posted to the AALS website authored by a non-standing committee of Deans.  The confusion arises because:

  1. Neither AALS nor this special Dean’s committee ever participated in the two stage TFARR process and so appear to be sort of “johnny come latelys, ” and
  2. The letter mistakenly focuses on an earlier draft of the final proposal failing to recognize the compromises already reached in the final proposal.

I understand that there are efforts underway to correct the confusion which makes me happy since the Deans’ letter is signed by two people whom I have long admired in a variety of contexts.

Other blogs are already exploring the 15 credit  proposal and its interesting and creative approach. For example,   “Kudos to California”  What do our readers think?

Building on Best Practices now available as eBook

Are you trying to:

  • Develop a meaningful law school mission statement?
  • Understand new accreditation requirements, learning goals, and outcomes assessment?
  •  Expand your experiential offerings?  Decide whether to use modules or courses?  An on-site clinic, an externship, or community partnership?
  •  Teach ALL of your students in the most effective ways, using a full range of teaching methods?
  • Add to your curriculum more of the professional identity, leadership, intercultural, inter-professional and other knowledge, skills, and values sought by 21st century legal employers?
  • Lead thoughtfully in the face of the challenges facing legal education today?

These and other topics are addressed in Building on Best Practices:  Transforming Legal Education in a Changing World,  now available in ebook format from LexisNexis at no charge.

The print version is not yet out.  LEXIS-NEXIS is taking advance orders for $50, plus shipping.  BUT we understand that they will make one copy available to every US legal educator for free upon on request.  Details on this and international availability still to come.

Thanks, and congratulations, to book project sponsor Clinical Legal Education Association (CLEA), the more than fifty legal educators who participated as authors, and the countless others who assisted as readers and in numerous other ways.

And, a huge shout-out to my wonderful and talented co-editors, Lisa Radke Bliss, Carrie Wilkes Kaas, and Antoinette Sedillo Lopez.

Building on Best Practices for Legal Education Manuscript Submitted to Publisher

Four editors,  59 authors, 92 readers, three copy editors, librarians from two schools, a secretary, miscellaneous consultants, three student assistants for bluebooking, and one for setting up perrmacc links.*

Many people, occasionally in multiple roles, were needed to produce the manuscript sent to Lexis last Monday for the forthcoming book Deborah Maranville, Lisa Radtke Bliss, Carolyn Wilkes Kaas, and Antoinette Sedillo López (eds.),  Building on Best Practices:  Transforming Legal Education in a Changing World. (Lexis 2015).  A monster project — but, as I assured a friend, no, not a manuscript about monsters and not monstrously unpleasant to produce – just big, ambitious, and sometimes exhausting for the editors and authors.  A big thank you to all who participated!

The book is a follow up to CLEA’s Best Practices for Legal Education, the 2007 volume by Roy Stuckey and others that inspired this blog.  Like Best Practices, this book will be distributed for free to legal educators.  Lexis has promised to make it available in electronic format through their e-book library and to provide print copies on request.  Look for it in four to six months — if all goes smoothly perhaps in time for the AALS Clinical Legal Education Conference in early May.

The coverage of Building on Best Practices is wide-ranging.  To quote from the Introduction, “[t]his volume builds on the call to link mission and outcomes; emphasizing the themes of integrating theory, doctrine and practice, developing the broader spectrum of skills needed by lawyers in the twenty-first century, and taking up the question how best to shift law school cultures to facilitate change.”

Advance praise for the book has included:

  • “[M]ilestone in legal education . . . that legal educators will rely on as much as . . . on the first Best Practices book.”  (Patty Roberts, William & Mary)
  • “Educational for folks who don’t know much about experiential education and insightful for those who do. . . .Really something to be proud of . . . an invaluable resource to schools as they go to work on implementing the ABA’s new requirements for learning outcomes and assessment. . .The perfect product coming out at the perfect time.” (Kate Kruse, Hamline)

Once again, CLEA deserves kudos for its support of an important scholarly project on legal education.  And the Georgia State University, University of New Mexico, Quinnipiac University, and University of Washington Law Schools deserve a big round of thanks for supporting the co-editors in this project.

https://perma.cc/ provides an archive for those annoying website links that quickly become outdated.

The Baby Has Finally Been Birthed!

Comprehensive revisions passed

The ABA House of Delegates passed the comprehensve revisions with “minimal  fuss” according to the ABA Journal linked  above.  One area, however, garnered  significant attention and also resulted in  an odd, though perhaps meaningless ,  procedural move.  The House voted  to send back to the Section on Legal Education for further consideration the comment to standard 305 which prohibits payment to students for credit-based courses.

What does this mean? Law schools which have not already done so must start identifying, articulating publicly and assessing student learning out outcomes, providing every student six  credits of clinic or clinic-like experiential courses and requiring students to take two credit hours worth of professional responsibility coursework.

Well, it’s a start……

Five Tool Lawyers

Leading Northwest legal practitioner and technology entrepreneur Marty Smith has an interesting post on the Five Tool Lawyer over at Legal Refresh. Using the metaphor of the Five Tool Lawyer, Marty breaks apart the stages of problem solving, incorporating risk analysis in a way I found helpful. In my response Five Tool Lawyers and Legal Education, I critique aspects of the Five Tool Lawyer metaphor for compressing too much into the 1st [Use interviewing skills to gather client facts, goals and needs] and 5th tools[Counsel, document, negotiate and advocate on behalf of client]. But here’s why I thought the metaphor was compelling:

"Compelling, because [it] moves beyond issue spotting v. problem solving to articulate the stages of problem solving, targeting a spotlight on often overlooked aspects. . . . By focusing on risk, the metaphor highlights two often neglected stages of the lawyer’s work – “use judgment to assess actual risks” and “problem solve for best way to meet client’s needs with minimal risk.” At the same time, it implicitly places the legal problem in the larger context of the individual’s life, or the business’s health. And it underscores the fact that lawyers need to know how to assess the significance of legal risks within that larger context."


Get every new post delivered to your Inbox.

Join 1,001 other followers

%d bloggers like this: