What am I up to now?

April 30, 2024

April, 2023

At this point, you’ve read much too much about LLMs. But this page is about my life, and much of my last month has been about LLMs. So deal with it.

I’ve probably talked more with ChatGPT in March than any individual person. When I downloaded my chat history, I’ve written tens of thousands of words, almost entirely about measure and social choice theory. That’s way more than I’ve written in letters, emails, WhatsApp, or Telegram.I might have spoken more words to some people this month than typed to ChatGPT. This is harder to count tho. Luis, Polina, and Ivan are the main contenders.

Despite this massive growth in words-typed-in-conversation, talking to ChatGPT has no substitution effect on my other conversations. Or rather, the conversations it replaced wouldn’t have happened anyway. Last month, I wrote

I have a crippling inability to ask stupid questions, which can leave me floundering. But ChatGPT can’t judge! I’ve taken to keeping a ChatGPT tab open in lecture alongside the slides, and asking it about points of confusion.

It’s possible that exam stress would have eventually brought me to asking these questions to my friends, but more likely I would have been left with some lesser search engine, like Google.

Unlike the most interesting examples floating around, I’m completely unable to hold a conversation with Bing. I don’t think this is due to the recent handicapping, either, but a personal fault. I had access to Bing for about three days before it was hobbled, and I couldn’t get it to go psychotic on me at all.Almost disappointing. Even now, Ethan Mollick is getting highly creative long-form content from Bing, while I struggle to provoke anything more interesting than the first few paragraphs of a wikipedia page.

The other tool I’ve gotten use out of is Poe. Far less supercilious than ChatGPT, Poe has an advantage in giving information and a disadvantage in giving explanations.

Because of that, ChatGPT remains my tool of choice. And especially since the middle of the month when I canceled dinner with three friends because “GPT-4 just dropped and I want to see if it has any thoughts about cats or consciousness,” I’ve been relying on it more and more.

My use case is, still, entirely learning economics. And the new model is a big improvement — ChatGPT(4) can prove things in a way ChatGPT(3.5) couldn’t. But it still has flaws. In order, here are some things ChatGPT(4) is good at.

1) Explaining concepts. Right now, instead of writing this, I should be trying to understand 2-state Markov processes for asset pricing. When I do get back to it, I’ll prompt ChatGPT with our professor’s explanation of the concept, and ask ChatGPT to expound on this explanation. It might be perfectly clear to Prof. Rick van der Ploeg why the safe rate is countercyclical in a Markov process, but it’s not to me. ChatGPT can break it down. This is a use case where ChatGPT(4) rarely makes mistakes, and is more willing to admit confusion. If anything, it errs in the other direction — it’s too easy for me to convince the model it made a mistake when it didn’t.This happened repeatedly while trying to understand the stochastic discount factor. After ChatGPT apologized and flipped its explanation a fourth time, I opened up Cochrane’s Asset Pricing book — ChatGPT had it right the first time.

2) Explaining notation. Economists are really really bad at this. We had five macro professors put five versions of the Taylor rule on the board with five sets of notation. ChatGPT has seen them all before, and is eager to explain each one.

3) Writing questions (and, less so, solutions). For macro, I have access to more than a decade’s worth of previous exams. For micro and metrics, tho, there’s only one previous exam — last year’s. We don’t have nearly enough practice problems to get a good sense of preparedness. But feed ChatGPT the problem set questions and it’ll write you more to practice with! Only problem — sometimes, these are unsolvable. ChatGPT doesn’t really know the difference.

4) Proving theorems. Really, this is the only one that matters. How long until we can ask ChatGPT, here’s a theorem on verifiable disclosure I just came up with, can you prove it for me? Or worseBetter? , how long until we can ask it to come up with a theorem, then prove it? We’re not there yet — and seeing how foolish ChatGPT(4)’s proofs occassionally are, I wouldn’t be surprised if the “come up with a theorem” bit is easier than the “prove it” bit.Call that ‘Ramanujan mode.’ Could and should write a whole post on this.

Zvi writes “The AI is doing a lot of pattern matching on [previously asked] questions. That means that if your question can be solved by such matching you are in good shape, if it can’t you are likely in bad shape.”Cf. this tweet. But if you think the AI is ‘retrieving’ a ‘known’ answer, you’re thinking about it wrong. For these last two use cases above, I’m asking ChatGPT to move outside of its LLM’s training data. For example: I prompted with a theorem from a recent paper Meg Meyer has with Edoardo Gallo, which is definitely not in the training data. While Meg could (and did) prove this theorem, ChatGPT(4) couldn’t. Or it tried to, but the thing it wrote was definitely not a proof.

But then I asked it to sketch four different approaches to proving this theorem. One of the sketches resembled Meg’s proof, and I prompted ChatGPT(4) to try to build out that sketch.This took a lot of trial and error, and I gave up before getting the model to produce a real proof. But if you want screenshots, reach out. The AI was able to build out a proof from there, with my input and nudges, much more easily than it could generate a proof ex nihilo.

ChatGPT won’t be replacing microeconomic theorists this month. But it has read more proofs than any individual academic, and if you’re struggling to prove something, asking the model for suggestions is a no-brainer. For most AI advancements, there’s a period where AI+Avg.Human will be better than either AI or the SmartestHuman. We’re not there yet,In the battle of GPT(4)+Me vs. Alex Teytelboym, Alex will still win every time. but I’d be surprised if even the best proof-writers couldn’t get value from an LLM.

Reading
Eating
Listening

Reading

March was my worst reading month since I’ve been keeping data. I didn’t read much, and much of what I did read, I didn’t like. Usually, I’m much better at abandoning books early on, but I failed completely last month. I’ll put the dislikes in a sidenote;•How to Live Safely in a Science Fictional Universe, by Charles Yu (a poor man’s solipsistic Hitchiker’s Guide) •Your Driver is Waiting, by Priya Guns (much too rough and shallow) •What’s Our Problem, by Tim Urban (this took you five years? will be a footnote on a great long-form career) the strong likes are:

•”The Written World and the Unwritten World,” Essays by Italo Calvino. A new set of non-fiction; superbly collected and translated. My favorite two essays were the eponymous and the first, “Good Intentions,” about the universally thwarted desire to read on one’s holiday. The last section is chock-full of Calvino’s reviews of books on science and anthropology. As all of these are more than fifty years out of date, I would never pick them up — but Calvino’s reviews are worth the time.Another delightful tidbit: ‘I love Chesterton because he wanted to be the Catholic Voltaire and I wanted to be the Communist Chesterton.’ I’ve always needed a Communist Chesterton!
•”Comedy of Errors,” by William Shakespeare. The play with the most consensus from critics. Impeccable farce; even as Shakespeare’s earliest play, there’s wit and beauty in the language, and just the right amount of mysticism. Hints of “Midsummer Night’s Dream” and “Othello” recur. There are no good performances online; recommendations are welcome!
•”Red Plenty,” by Francis Spufford.
•”The Doomsday Machine: Confessions of a Nuclear War Planner,” by Daniel Ellsberg. I was introduced to Daniel Ellsberg by Arundhati Roy’s 2016 book “Things that Can and Cannot Be Said” — Roy and Ellsberg traveled to Moscow to sit down with Edward Snowden. Roy had an idea for a great conversation between the two great American whistleblowers. That version of Ellsberg wasn’t an ideologue, just a disappointed academic. Ellsberg’s passion was never the Vietnam war; that was just a tragedy he was forced by conscience to interact with. Nuclear policy was his true passion. In “Things,” Ellsberg asks Snowden:

You get the feeling that “Doomsday Machine” is the book which Ellsberg has wanted to write for his whole life, if he hadn’t been drawn in by Vietnam. This version of him is an ideologue; he mentions, offhand, the weeks he spent hiding from the FBI, attempting to hide, uncover, and re-hide documents about US nuclear war plans. He has devoted much of his post-Vietnam work to reducing the risk of nuclear war, with measurable success. I hope to review the book in full soon.

Ellsberg will likely die this year.

“When I copied the Pentagon Papers in 1969, I had every reason to think I would be spending the rest of my life behind bars. It was a fate I would gladly have accepted if it meant hastening the end of the Vietnam War, unlikely as that seemed (and was). Yet in the end, that action – in ways I could not have foreseen, due to Nixon’s illegal responses – did have an impact on shortening the war. In addition, thanks to Nixon’s crimes, I was spared the imprisonment I expected, and I was able to spend the last fifty years with Patricia and my family, and with you, my friends.”

We will miss him.

Monthly Le Guin

I read The Dispossessed sometime in college,Unlike the supremely precocious ML, who read it at the age of seven. and at least three years after first reading “The Day Before the Revolution.” “Revolution” is a short story prologue to Dispossessed, published the same year, and stands well enough on its own. It’s simpler than most Le Guin — most of the action takes place in a “nice large, sunny, room, proper for a drooling old woman who had started a world revolution,” but sizzles nonetheless.

Odo, the aged revolutionary anarchist whose ideology inspired the anarchist society of The Dispossessed, also sizzles precociously. Facing the mounting revolution which will realize her dreams, her thoughts turn to lost loves, and death, and sex.

“Decency be damned. Even six months ago, before the stroke, she had made men look at her and like to look at her; and now, though she could give no pleasure, she could please herself.”

Odo, of course, never sees the world she makes for the dispossessed. It’s a surprise we the readers get to meet her at all — in her forward, Le Guin is also surprised when “Odo came out of the shadows and across the gulf of Probability, and wanted a story written, not about the world she made, but about herself.” One has to expect Odo would have been disappointed. Not out of any failure from the young anarchists she leaves behind in this story, but out of an innate contrarianism she inherits from Le Guin. Satisfaction be damned, as well.

Eating

Optimized for efficiency this month, except for one Michelin star.

Listening

• North African music doesn’t miss — my favorite album last year was “Music from Saharan WhatsApp” from Sahel Sounds. This month, “GOJO” was another revelation. A collaboration between traditional Ethiopian group Fendika and woodwind duo K-Sanchis (whom you might know as Trio Kazanchis), GOJO is exuberant. I’ve heard Steve Buchanan’s saxophone before, but never resonant like this. Loads of fun.
• At Mallams in Oxford, I saw a mandolin in the window. While I ended up bidding more than twice the auctioneer’s estimate, someone else went home with the gorgeous thing. So I won’t be learning to play the lute, but I did go on a Chris Thile kick this month. Nickel Creek holds a special place in my heart,You may as well watch their Tiny Desk over again. but I listened to Thile’s Bach Sonatas and Partitas over and over again. He’s something special and worth cherishing.
• “Held Together,” by Aberdeen. Funky, prog-rock, horn ensemble. Very dancey.