AGI Chronicles

Cheery Friday Greetings from Barb Oakley! (Friday ran long this week.)

What I’m Reading: The AGI Chronicles

I’m in the middle of Kevin Roose’s new book, The AGI Chronicles: The Inside Story of the Race to Create an Artificial Superintelligence—it is absolutely riveting. Roose covered the world of AI for The New York Times for years; for this book he interviewed more than 150 key players at OpenAI, Anthropic, and Google DeepMind. And so we learn the inside story of why Anthropic’s founders left OpenAI, of Sam Altman’s firing and rehiring, and of the officials in Washington who are trying to regulate a technology that changes faster than they (or we) can keep up. Roose compares the whole race to the Manhattan Project, but personally, I think it’s even bigger than that.

(Side note, speaking of the Manhattan Project, in the months right after World War II ended, my father flew atomic bomb test materials from Roswell, New Mexico, to Johnston Atoll in the Pacific—about twenty-two hours in the air, flown in stages. He did, that is, until one day the landing gear wouldn’t lock down after takeoff, so nobody knew whether the plane would crash when it tried to land. He had to circle Roswell for some twelve hours to burn off fuel, so that if the gear folded, the plane would at least be light and nearly dry when it hit. That’s a lot of time to think about maybe getting a different career—which is how he became a veterinarian. And how I learned interesting stories about Roswell, where my mother was from… Another time!)

Anyway, back to The AGI Chronicles. As it turns out, a surprising amount of Roose’s book is about learning. The labs train these models in part through reinforcement learning. Reward the model’s good answers and penalize its bad ones—this creates what is called a reward prediction error, meaning the difference between what the model expected to get and what it got. In 1996, Read Montague, Peter Dayan, and Terry Sejnowski (yes, our Learning How to Learn co-instructor) proposed that dopamine neurons in the brain compute that same reward prediction error. Their idea became the standard textbook account of what dopamine does. Thirty years later, neuroscientists are finding that the brain’s version is a good deal messier than the algorithm.

A Dilly-Dally in Dopamine

For a couple of decades, the textbook explanation was that dopamine cells in the midbrain fire a quick burst that boosts learning when something turns out better than expected. The underlying background level of dopamine was thought to set how motivated you are.

Josh Berke, a neuroscientist at the University of California, San Francisco, wrote a review in 2018 titled “What does dopamine mean?” that explains why that story is in trouble. The dopamine cells sit in the midbrain, but they deliver their dopamine through long fibers that end in the striatum, and the amount that comes out at those endings is decided locally. Think of a city water main running at steady pressure. What arrives at your sink depends on the faucet, and in the case of the brain, the faucets are in the striatum. Neurons there can open the faucet wider as a reward gets closer, without the midbrain pumping any harder. This is why dopamine measured in the striatum climbs steadily as a rat works its way toward a treat while the midbrain cells show no such climb. Berke thinks dopamine is best understood as the brain’s running estimate of whether what you are doing is worth spending your energy, attention, or time on. The learning part of what dopamine does, the burst that fires when a reward turns out better than expected, is still the same as researchers have long thought. That burst strengthens whichever connections in the striatum were active in the second or two before it arrived, the ones that did the work that led to the reward. Berke’s phrase for the two roles is that motivation “looks forward” and learning “looks backwards.”

What does this mean if you are learning, say, vocabulary in a new language? A word gets the dopamine credit only when you retrieve it yourself and the retrieval comes a little harder than expected; glancing at the word again a minute later is too easy to earn the burst. The human studies say to retrieve a new word correctly three times the first day, with other words in between, and then to relearn it on about three later days, with each gap wider than the last—a day, then several days, then a few weeks.

A Free Memory Workshop with Nelson Dellis, November 5

The striatum came up in this newsletter back in April, when I wrote about the brain scans of my long-time friend Nelson Dellis, the six-time USA Memory Champion. Researchers at Washington University in St. Louis found that Nelson’s caudate nucleus, which is part of the striatum, had built connections to his memory networks that did not exist in any of the nearly 900 people they compared him with. He built those connections through years of daily practice. On November 5 at 1:00 p.m. Eastern, Nelson is giving a free live workshop on the three mistakes that keep people from remembering what they see, hear, and read, even when they are paying full attention, and what to do about them. Register here for Nelson’s free November 5 workshop.

Becoming an Expert Teacher—the Recording Is Up

Back in August I mentioned that my friend Nidhi Sachdeva, chair of researchED Canada and co-author with Paul Kirschner of their terrific book Becoming an Expert Teacher: Deliberate Practice for Effective Teaching, was giving a free McGraw Hill webinar. Some 18,000 people signed up. The hour-long recording is now open to everyone. Nidhi draws her lessons from Captain Chesley Sullenberger’s 2009 emergency landing on New York’s Hudson River, which shows what deliberate practice buys you when there are only seconds to decide. Then she asks teachers to think about the spontaneous decisions they make every day that shape what their students learn. Check it out! Knowing What to Do When You Don’t Know What to Do: Becoming an Expert Teacher

Nidhi and Paul also write a series of posts on expert teaching. Their latest one takes up the old argument about whether teaching is a science or a craft. Of course it’s both. The science is what a teacher needs to know going in—the subject, how people learn, how to teach, what tools to use, and how to relate to students. The craft comes from using that knowledge in a real classroom: “the growing ability to judge when to slow down, when to press on, when to model again, when to hold back, when to question more, and when to let silence do its work.” The craft of teaching has to be developed over time, with proper mentoring and feedback on practice. Read it here: Is Teaching a Science or a Craft?

researchED Toronto—This Saturday

This Saturday, October 24, I’ll be at researchED Toronto, working one math example all the way down to the brain: the different jobs of declarative and procedural memory, what scaffolding physically consists of, and how JUMP Math teaches division. Nidhi is the host. If you’ll be there, come find me!

That’s all for now. Happy learning!

Barb Oakley

• Learning How to Learn—one of the world’s most popular MOOCs, with its recommended text, A Mind for Numbers
• Uncommon Sense Teaching—the book and Coursera Specialization!
• Mindshift—the book and MOOC
• Learn Like a Pro—the book and MOOC
• For kids and parents: Learning How to Learn (for Youth)—the book and MOOC. Pro tip—watch the videos and read the book together with your child. Learning how to learn at an early age will change their life!