Making It Stick II

I wrote an article last week about advice for students in Make It Stick: The Successful Science of Learning, by Peter C. Brown, Henry L. Roediger III, and Mark A. McDaniel.

The book also contains sage advice for teachers, much of which is especially pertinent to law school educators and administrators. With declining law school enrollments and falling bar passage rates, the American Bar Association in 2014 revised its Standards and Rules of Procedure for Law Schools relating to learning outcomes, credit hours (including provisions for student out-of-class coursework) and objective policies and procedures for determining the credit hours to be awarded for coursework. Making It Stick describes dozens of practices to help legal instructors remove obstacles to learning, facilitate understanding, and promote success in achieving student learning outcomes.

They advise teachers to (1) explain to students how learning works, (2) teach students how to study, (3) create desirable difficulties and provide a high level of structure in the classroom, and (4) be transparent. I incorporate all of these practices into my classes.

1. Explain to Students How Learning Works.

Students often arrive at law school with the belief that you either have innate talent or you don’t. This fixed mindset is generally reinforced by the first year curriculum. Most first year first semester courses involve a steep learning curve that requires students to learn to read cases, extract legal rules, reason by analogy, identify legal issues in complex fact patterns and write coherent, persuasive analyses about those issues. While the first year student may get feedback in her legal writing class, most first year classes are taught in lecture format or using the Socratic method. The students have no interim assignments and receive no feedback to help them evaluate whether they are on track. Only with the release of the first semester grades do students receive any concrete information about whether they have learned the material to the necessary standard, mastered the essential skills to succeed in law practice, or even attain the knowledge base pass the bar exam. At that point they are often well into the next semester with its new demands and feel like they do not have the time to re-hash their past performance even though this is the most important part of the learning process.

At this point, their perceptions of themselves as either the ones who have “got it” or the ones who “don’t” begins to harden. The ones who have “got it” sometimes begin to avoid challenges that would draw into question whether they have “got it” (or negatively impact their GPA) and the ones who believe that they “don’t” sometimes lose faith that hard work will get them anywhere.

As the authors of Make It Stick explain, students need to be taught at the very beginning of classes that:
• Learning requires that chemical changes take place in the brain. The more effort and the more challenge, the more effective the learning process.
• When learning is easy, it is often superficial. It is also easily forgotten.
• We learn more effectively when we wrestle with new problems before being exposed to the solution. Even though it’s emotionally unsettling, it’s also more effective to think hard about how you’d answer a question before you have access to the answer.
• Learning necessarily involves setbacks and failures. This feedback is often precisely what we need to adjust our strategies and to achieve mastery.
• To achieve mastery, we must surpass our current levels of ability.

Therefore, I share excerpts from The New Science of Learning and Make It Stick with my students at the beginning of the semester and connect that information about how we learn with the organization of the assignments, the graded exercises and exams, and the learning outcomes identified in my syllabus. I especially encourage students to adopt a growth mindset. As Henry Ford said, “Whether you think you can or you think you can’t, you’re right!” I explain that struggle increases your intellectual abilities in the same way that exercising or lifting weights increases your physical strength.

2. Teach Students How to Study.

Students may arrive in law school with the belief that re-reading the texts and highlighting or underlining key material is sufficient engagement for them to remember and use the material. They may have habits of procrastinating and then pulling an all-nighter and cramming to meet a deadline. These practices are, of course, ineffective in professional education. First, cramming doesn’t last. The material is not retained for the long term. Second, rote memorization does not help students apply the material, which is what they’ll be asked to do for their course exams and for the bar exam. Third, distributed learning is far more effective. Student recall and mastery improves when questions are asked several times over increasing periods from the date the material was first presented and when different topics or problem times are alternated. Fourth, interacting with the material is the key to learning. When students are required to perform some task associated with the material they are more likely to remember it. If the practices are identical to key skills they will be using in a professional capacity, then the students can see their progress over time and master practices that will ensure their long-term success.

3. Create Desirable Difficulties and Provide a High Level of Structure in the Classroom.

As the authors of Make It Stick explain, high structure classes with weekly and daily low-stakes exercises provide students with opportunities to practice their skills. The best book I have read on classroom structure was written for elementary school teachers. Nevertheless, The First Days of School: How to Be an Effective Teacher, by Harry K. Wong and Rosemary T. Wong, provides a succinct and valuable set of guidelines for anyone who has begun their teaching career without any prior formal training.

Other effective classroom practices promoted in Make It Stick include requiring the students to generate answers to problems they do not currently know how to answer, and asking them to elaborate, to remember older material and connect it to new material. Finally, spacing out the material, interleaving the material, and varying the topics and the problems has been very effective in enhancing student learning. All of these practices reinforce the neural pathways of learning.

In my classes, I have weekly assignments. In a class like Income Taxation, students must learn several essential skills as well as learn key material. I have divided the course into units based on the types of material to be learned. I assign sections of the tax code and regulations that the students must read. This is challenging, especially for students who have completed only the first year curriculum, which case-based and seldom involves statutory reading and interpretation. To facilitate this work, I have developed worksheets that ask students specific questions about each code section or regulation and direct the students’ reading. By the end of the course students often express surprise at how much their skills have improved in reading the code and the regulations. It’s gratifying for them (and for me) to see how daily practice can so quickly lead to accomplishment.

I also assign tax problem sets each week. I ask the students to attempt the problems and I give them credit for giving the homework a good faith effort and turning it in. As the authors of Make It Stick explain, students learn more effectively when they attempt to do the problems before they’ve been shown how to solve those problems. They are also more attentive when I go over those problems in class.

Finally, as the authors of Make it Stick indicate, while high stakes tests can be stressful and anxiety provoking, students accept quizzing when the stakes are low and when they know in advance the dates the quizzes are going to occur. I give multiple choice clicker quizzes either at the end of a unit in class or in review sessions I provide every other week. When the technology is available, clicker quizzes give students anonymous feedback about their performance. Students want to see how they are doing, but not at the expense of embarrassing themselves in front of their professor or before other students. Clicker quizzes also allow the students to contextualize their performance by giving them feedback about how the rest of the class is doing as a whole.

Spacing out the material, interleaving the material, and varying the topics and the problems has also been very effective in enhancing student learning. The review sessions, which I hold every other week, give the students another chance to apply what they’ve learned in prior weeks. By working through another round of problem sets or past mid-term or final exams, they reinforce their learning and develop a higher level of mastery. Occasionally, I will send students to the blackboard to let them collaborate on the review problems.

4. Be Transparent.

Finally, Make It Stick encourages teachers to be transparent. The syllabus contains the information on their weekly assignments, information on way they will be graded, and a rubric with respect to class participation and how that will be evaluated. When I give midterm examinations and finals, I post prior exams and at least one example of the rubric I used in grading that exam or a copy of the “best of” student answers that I delivered to the prior class. When I grade, I use a rubric and key the student points to the rubric. After the exam, I either go over the rubric in class or provide a copy to the students so that they can compare their answers to the rubric.

Finally, reflection is important. It’s important for professionals to use the feedback they’ve received to improve their performance over time. I ask students to spend a few minutes reflecting on what they learned from their exam and ask them “What would you do differently next time?”

These reflection questions are the most important part of every semester for me when I evaluate my own teaching. As I grade my students’ exams I ask: “What did all of the students get correctly? What did most of the students miss? How can I explain this more clearly? What problems can I assign that will help the students master this material?”

This, and the information I receive on the student feedback forms, is what sticks with me.

Making It Stick

Samford University recently delivered a copy of the 2014 hit "Make It Stick: The Successful Science of Learning, by Peter C. Brown, Henry L. Roediger III, and Mark A. McDaniel to my office. Since I'd bought a copy in 2014, I concluded that this was a signal that I should actually read it. . .   I now share key ideas of the book with my students.

As this recent Business Insider article notes, most people have never been taught how to learn. Many students fall into the "either you got it or you don't" fixed mindset and assume that if a subject is not easy for them, it's not something they can learn. This book (as well as books like The New Science of Learning) helps teachers and students understand the process of learning so that they may work with and enhance that process instead of creating additional obstacles to learning or giving up altogether.

The latter chapters of Make It Stick have a number of tips for students (which I share with my classes). Some of the high points are:

1.    Practice retrieving new learning from memory. We live in an era where so much knowledge can be accessed via Google on your cell phone. We've gotten out of the habit of remembering facts and details. However, the pace at which we can retrieve information may determine whether we can make the necessary connections to solve new problems. By practicing retrieving new memories we cement those memory paths and strengthen our recall. Taking a tip from Mary Pat Wenderoth, a Biology professor featured in the book, I've reminded my students when we are doing review, to take a few moments to try to remember before turning to their notes and books for the answers.

2.    Space out your retrieval practice and "interleave" the study of different problem types. We find some level of gratification in cramming and then perfecting our retrieval of one set of things before going to the next set of things. Unfortunately, this gives us the illusion of learning without really achieving much. Instead, by letting some time lapse before we try to recall, we improve our learning and memory of the material. Just as we build strength more quickly when we give our muscles a day to rest after strength-training exercises, we enhance our learning when we take time to rest or focus on other materials. Likewise, when we study different things, using different skill sets or methodologies, we may feel that we are learning more slowly, but we are learning more effectively. Again, we are strengthening the pathways to new knowledge.

3.    Elaborate. "Elaboration" in this context means relating the new things you've learned to what you already know. When we elaborate, we connect the new pathways, making it more likely that we will actually take the new pathways when facing the next problem.

4.    Generate. When we generate answers to problems we don't currently know how to solve, we take a tour of older problems that we have been able to solve and consider whether those solutions may work. We may also try to come up with new ways of addressing those problems. When we try to answer problems before we have the process and the skills in place, or access to the answer key, we have again strengthened old pathways and primed our attention to learn something new.

5.    Reflect. When we take a few minutes to ask what went well, what could have gone better, and what we could try differently next time, we professionalize our learning process. This is activity that many professions undertake. Medical communities routinely discuss their diagnoses, patient results and their mistakes in an effort to improve their analysis, process, teamwork and patient outcomes. Athletes review film and discuss their past competitions with their coaches to improve their next performance.

6.    Calibrate. Calibration means that we are checking our results against a standard. If we think that we have performed fairly well, but have not examined our performance against a model or a rubric, or gotten feedback about whether we've met the standard, we may be deluding ourselves. you may and responses against a rubric or a "best of" student answers from a prior exam.

7.    Mnemonic Devices. We have all developed acronyms for remembering essential information. The ancient Greeks were able to recite their epic tales of heroism and woe by memory by constructing a house in their mind. They begin the epic poem by "walking" into the foyer of the house and using the objects they imagine they see to evoke the first opening lines. They would progress through the poem by moving into the other rooms and using the sequence of items seen to recall the subsequent lines. Oxford students call these mental edifices "memory palaces."

Many of my students have embraced these ideas and begun to trust the scientific research behind the techniques. When they achieve success in their classes, however, is when it really sticks!

Flow - The Secret to Happiness (and to Success)

Mihaly Csikszentmihalyi grew up in Eastern Europe. During World War II, when he was between the ages of seven and ten, Csikszentmihalyi noticed how many of the adults he knew were unable to recover from the privations of war and reassemble their lives. He began to wonder what was necessary for human life and happiness. He was inspired to study psychology after hearing a lecture by Carl Jung in Switzerland. He emigrated to the United States at age 22 and graduated from the University of Chicago with a B.A. and a Ph.D. in psychology and later served as head of the department of psychology there. He now works as the Distinguished Professor of Psychology and Management at Claremont Graduate University.

Csikszentmihalyi developed the theory of "flow," a mental state of total absorption in an activity in which one finds fulfillment and joy. Athletes often describe the state as being "in the zone." In law school, the two times I received the highest grade in my class (and American Jurisprudence Awards), I knew I would do so. I walked into the exam in a "flow" state.

Here's his TED Talk on Flow - The Secret to Happiness.

How does it feel to be in flow?

1. Completely involved in what we are doing - focused, concentrated.

2. A sense of ecstasy - of being outside everyday reality.

3. Great inner clarity - knowing what needs to be done, and how well we are doing.

4. Knowing that the activity is doable - that our skills are adequate to the task.

5. A sense of serenity - no worries about oneself, and a feeling of going beyond the boundaries of the ego.

6. Timelessness - thoroughly focused on the present, hours seem to pass by in minutes.

7. Intrinsic motivation - whatever produces flow becomes its own reward.

To optimize learning and academic and professional performance, we all must seek flow. Fortunately, Csikszentmihalyi has developed a number of resources to help us gain access to flow. Csikszentmihalyi's books may be found here.

Bibliography on Teaching and Learning

Over the past twelve years I have learned about teaching from mentors, from sitting in on other law professors' classes, from attending the George Mason Law and Economics Institute for Law Professors (where I not only learned more about economics, but had the great privilege of watching some truly superb teaching), from reviewing my student teaching evaluations and from interviewing students about what worked and what didn't work over the course of the semester.

In addition to all of this, I've read extensively. Here are a few of the resources I've found very helpful in improving my teaching and enhancing students' ability to learn.

Neuroscience of Learning 

Terry Doyle and Todd Zakrajsek, The New Science of Learning, How to Learn in Harmony with Your Brain  (2013).

Terry Doyle, Helping Students Learn in a Learner-Centered Environment (2008).

Peter C. Brown, Make It Stick: The Science of Successful Learning (2014).

Daniel Kahneman, Thinking Fast and Slow (2011)

Addressing Emotional and Intellectual Blocks to Learning

Amy Cuddy, Presence (2015).

Susan Cain, Quiet (2012).

Brene Brown, Daring Greatly (2012).

Carol S. Dweck, Mindset: The New Psychology of Success (2007).

TED TALKS
Angela Lee Duckworth, TED Talk on The Key to Success: Grit

Brene Brown, TED Talk on The Power of Vulnerability 

Brene Brown, TED Talk on Listening to Shame

Susan Cain, The Power of Introverts 

          Ariana Huffington, How to Succeed? Get More Sleep

Exploring The Joy of Learning

          TED TALKS

Shawn Anchor: TED Talk on The Happy Secret to Better Work

          Mihaly Csikszentmihalyi, Flow, The Secret to Happiness

Technology and Learning

NPR: Attention Students, Put Your Laptops Away

Pam A. Mueller & Daniel M. Oppenheimer, The Pen Is Mightier Than the Keyboard, Advantages of Longhand Over Laptop Note Taking, 25:6 Psych. Sci. 1159 (June 2014).

The Importance of Sleep

Ariana Huffington, The Sleep Revolution (2016).

Judah Pollack and Olivia Fox Cabane, Your Brain Has A "Delete" Button—Here's How To Use It, Fast Company (May 11, 2016).

Law Teaching Best Practices

Michael Hunter Schwartz, Gerald F. Hess, Sophie M. Sparrow, What the Best Law Teachers Do

Terry Doyle, Learner Centered Teaching, Putting the Research on Learning into Practice (2011)

Ken Bain, What the Best College Teachers Do (2004)

Kent Syverud, Taking Students Seriously, a Guide for New Law Teachers, 43 J. LEGAL ED. 247 (1993).

Lectures and Oral Delivery

Bill McGowan and Alisa Bowman, Pitch Perfect, How to Say It Right The First Time, Every Time

Diagrams, Drawings, Presentations and Visual Delivery

Lee Lefever, The Art of Explanation

Organization and Structure

Harry K. Wong and Rosemary T. Wong, The First Days of School: How to Be an Effective Teacher (Book & DVD) 4th Edition

Statement of Teaching Philosophy

While drafting a "statement of teaching philosophy" is a standard process for job applicants in colleges of arts and sciences in the United States, it is somewhat more rare for a law school to ask for one. Fortunately, more law schools are requiring them of their entry-level job applicants, forcing prospective law professors to think about what they are doing, why they are doing it and how they propose to go about the teaching. Perhaps a statement of teaching philosophy is more important for law professors than for professors in other fields because there is nothing in law school or in practice that prepares you to become a law professor.

This is a subject on which I've been ruminating for over 25 years. From the time I first read Duncan Kennedy's "Little Red Book" before attending law school, to my receipt of my most recent student evaluations, I have been asking myself whether legal education is helping students develop the skills, habits and ways of thinking about the world that will transform them into competent lawyers, honorable and engaged citizens and effective leaders. In Legal Education and the Reproduction of Hierarchy, Kennedy argues that the lecture format, the use of the Socratic method, the examination process, and the way law school students are socialized undermine initiative, independent thinking, agency, and assertiveness and slot them in specific trajectories that have no purpose other than replicating the existing economic and social hierarchies. Having spent the last 30 years working in one capacity or another in the legal world, I have to conclude that he is correct in many respects.

Fortunately, the upheavals of the last 20 years in the legal profession and of the last five years in legal education have prompted a broader discourse about the goals and functions of legal education. This inquiry is bearing fruit in a variety of ways, but I think one of the more positive developments is the move to incorporate science-based research on learning, emotional and intellectual blocks to learning and effective and innovative teaching technologies. I have incorporated these developments into my own Statement of Teaching Philosophy:

Neuroscience research has transformed our understanding of teaching, learning, and intellectual development over the past 20 years. The central message from this research is straightforward: "The one who does the work does the learning."[1] Drawing from this body of nationally recognized scholarship in education,[2] I have created courses that engage students as partners in the educational enterprise. I structure the readings and assignments to build student confidence, expand their capacity to manage uncertainty, promote persistence, and reward effort and independent thought. I also try to make the courses fun; when students enjoy class, they work harder and perform at a higher level.

For doctrinal courses, including Income Tax, Partnership and Corporate Tax, Environmental Law and Property, I divide the material into weekly modules to help students manage their workload and expectations. First, after a short introduction to the material, I introduce students to close reading and interpretation of the cases, statutes and the regulations. I have developed statutory worksheets help students to focus their attention, extract the main legal concepts, understand the architecture of the law, and develop an outline of authorities. The worksheets also facilitate group discussion about the incentive structures and policies undergirding the law. Second, I require students to turn in weekly problem sets. The problems give the students the opportunity to apply what they have learned in a fact-rich, client-based context. Third, I test basic comprehension with multiple-choice quizzes using TurningPoint or i-Clicker student response systems or the TWEN Instapoll function. The software aggregates students’ responses in bar graph format so that they can tell whether their answers are in alignment with those of their peers. They provide an opportunity for peer-to-peer learning and allow me to address any broader confusion. Fourth, I often give mid-semester examinations so that the students gain important feedback about their performance. I have sometimes provided skeleton outlines of the material to help students see the broader picture; the outlines tend to even the playing field and reduce the advantages associated with course notes that are transferred selectively through social networks.

For Tax Policy, Climate Change and the Law, and other seminar courses, my teaching has been strongly influenced by my own coursework. Michael Sandel’s lectures on distributive justice at Harvard initially inspired me to pursue a career in the law, to dedicate myself to pro bono and nonprofit work, and to combine my experience in private practice with my interests in social and environmental policy. During my LL.M. work at New York University, Daniel Shaviro’s and Alan Auerbach’s Tax Policy Colloquium and Lily Batchelder’s Tax and Social Policy course introduced me to economic analysis. In my own courses, we examine incentives, public policy goals, and legal and economic structures designed to alter those incentives or overcome transaction costs to facilitate trade of entitlements. We discuss whether the law is effective, whether policy-makers’ goals have been met, whether there are significant spillover effects, and how legal structures might be modified to better achieve those goals with lower social or budgetary costs. I also employ some of the teaching methodologies I learned at Harvard and NYU, including requiring students to lead class discussions individually and as a part of a group effort, critique the assigned readings, and develop a research and writing plan. I have also asked students to provide constructive feedback for their peers’ drafts; students learn to improve their own work by critiquing the work of others.

For skills-based courses, such as real estate transactions, I would accompany the readings with a simulated transaction in which the students draft, review and edit client engagement letters, consider conflicts of interests, perform due diligence investigations, draft, negotiate and edit real estate documents, and close the transaction. For some courses, we may also discuss alternative deal structures, and identify the benefits, costs and risks associated with each option. My goal is to familiarize the students with the transactional process, grant the students a solid foundational knowledge of the documents and the economic and legal issues that may arise, and develop their skills to negotiate and resolve those issues to their clients’ advantage.

Over the past two years I have also been working closely with Professor Laurie Zimet’s AcademicSupport Program to improve student performance in first year classes and prepare them for the bar examination. Following an in-depth training with Professor Zimet, I have supported students in skill-building workshops and one-on-one meetings focused on helping students identify their academic strengths and weaknesses, build their skills and confidence, and explore their other concerns that may have been interfering with learning.

Finally, I know from both my in-class experiences, and from my work with pro bono attorneys and pro se clients to expand access to justice in Georgia, that technology is a powerful tool for engaging and empowering students. I have employed Blackboard, TWEN, Canvas, audio, video, and student response systems to engage students and to accelerate the learning process. Once my course responsibilities stabilize and I can obtain access to user-friendly videotaping and editing software, I plan to use the “flipped” classroom methodology developed at West Point. In flipped classrooms, the students watch the introductory lecture prior to class, complete and online quiz and then use classroom time for problem solving, skills development work, and policy discussion.

---

[1] Terry Doyle, Helping Students Learn in a Learner-Centered Environment 63 (2008).
[2] I have read and incorporated best practices from a number of books and articles about how students learn and what constitutes good teaching. My growing bibliography is HERE. I have attended a number of conferences and workshops on learner-centered teaching and incorporated their suggested practices into my curriculum: Southeastern Association of Law Schools Annual Conference, New Law Teachers' Workshop (Aug. 6 - 7, 2013); University of Louisville Delphi Center for Teaching and Learning Events: "Becoming a Learner-Centered Teacher" (Feb. 7, 2013); “What Makes a College Teacher ‘Great’? Learning to Make the Most of Your Teaching Activities” (Sept. 29, 2011); “Small Steps are Best: Incorporating Critical Thinking & Assessment into Courses One Step at a Time” (Apr. 11, 2011). I have also observed the classes of Tony Arnold, Tom Blackburn, John Crawford, Bill Dodge, Susan Kuo, Jack Miller, Roger Park, Radhika Rao, Joel Samuels, Reuel Schiller and Laurie Zimet. Finally, I've taken an improvisational theater course to improve my extemporaneous speaking and classroom performance.