What am I up to now?
October, 2026
Contents
Updates
Happy fall from San Francisco! The view from the center of AI hype has started to escape containment, first with worries about agentic AI hackers and soon after with concerns over existential risk, anti-trust regulation, and court cases. Friends, family, and old bosses have called to ask me about AI in the past few weeks more than ever before.
One of the common sticking points is a methodological question about subjective probability, which has been a fascination of mine. Jacob Coxon, who started off this preference cascade, referenced a 10% chance that AI would kill us all in the next decade. When someone asked a bunch of AI researchers in 2023, the median estimate was 5% chance for extinction-level events; when someone else asked a bunch of superforecasters their median estimate that AI would kill us all was 3%. There are a few reasonable responses to this. One of which is “AHHHHH”, and another is:
10%? 5%? What does that even mean? How can you have a 10 percent chance of something like that happening? Joseph, you’re talking nonsense. Stop it. Percentages should be used for dice, not for stuff like this.
The question of what subjective probability estimates mean has never really made sense to me. However, I do think it is still a useful tool. One of my favorite pieces of writing on this comes from the proceedings of a 1971 conference in Armenia on communication with extraterrestrial intelligence (CETI). It was put on by Carl Sagan, and attended by many of the most famous scientists from the USSR and the United States. One of the sticking points at this conference was also subjective probability estimates. What does it mean when the Drake equation puts a percentage chance on life arising on another planet?The usual estimand of the Drake equation is “number of active civilizations in our galaxy”. Most of the inputs are not probabilistic! E.g., “rate of star formation” is pretty well nailed down based on observing our galaxy. What Drake and Sagan and everyone else at the conference only knew probabilistically were how often life, intelligence, and advanced technology emerged. For that had only been observed once before.
At the conference in Armenia, Francis Crick, the co-discoverer of DNA, put forward the view that life emerging was like being dealt one particular hand of cards from a deck of unknown size.
We know that this is a rare event and it is not reasonable to try to estimate the probability of the event simply because it has happened to us. … But we only have a unique event and strict probability theory says that we are not allowed to deduce probability in that way.
Instead, we turn to subjective probability: “the capacity of the human mind to see relationships between things, out of past experience, which are not clearly formalized.” This is where estimates of extinction risk from AI and other speculative events come from: inferences, connections, intuition, and finally subjective estimates. You could try to estimate how many computations it took evolution to “invent” human intelligence, just ask a lot of smart people, or break the question into six smaller guesses and multiply them togetherwhich is a Drake equation for AI. Carlsmith got about 5% this way, then later raised it above 10%.
My favorite thing about subjective probability estimates is the endless potential for nitpicking. What counts as a neuron? What does it mean for a planet to be able to support life?Sagan at the conference: “There is a famous book published about 1912 by Lawrence J. Henderson called The Fitness of the Environment, in which Henderson concludes that life necessarily must be based on carbon and water, and have its higher forms metabolizing free oxygen. I personally find this conclusion suspect. If only because Lawrence Henderson was made of carbon and water and metabolized free oxygen. Henderson had a vested interest. Can we make some sort of objective judgement free from chauvinism, independent of our prejudices?” Both the Drake equation and Coxon’s 10% are open to plenty of nitpicks. This is Sagan’s reply, that he and Crick are just “playing different card games.” Crick has no truck with it. Crick’s take is that it’s all useless.
But in our present problem we do not have the experience on which to form these hunches. Therefore, in my view it is not legitimate to appeal to subjective probability and we find ourselves, therefore, I say, in a state of ignorance which we hope the future will reduce.
My favorite part of the conference proceedings is an appendix written later by Terrence Fine, who wasn’t at the conference but had been cited as a defender of subjective probability. Like Crick, he thought that any attempt at statistically reasoning about aliens was doomed by insufficient data. But given that challenge, subjective probability is “at present the only basis upon which probability statements can be made” about extraterrestrial life. Their purpose is only
decision making and the interpersonal communication of individual judgment and not, say, assessing the “truth” of propositions.
The outcome of the Drake equation is not a fact about the world, like the probability of drawing a straight flush is a fact about the world. A 10% chance of AI killing us isn’t a fact about the world. In the 1970s, it was interpersonal communication. Today it’s that and slightly more, due to the rise of interpersonal bets and prediction markets. Most importantly it’s something you can argue about:
While subjective probability statements are indeed personal, they are not arbitrary. … While it is not possible to criticize any single subjective probability assessment it is possible to criticize a collection of such assessments.
Nobody can show that a 10% chance of extinction is wrong like they can with a deck of cards. But anyone can nitpick everything that comes along with it. Each subjective probability comes with an associated collection of smaller guesses, some of which are backed by objective facts (the rate of star formation! the amount of compute in the world!) and some of which are just intuitions all the way down (number of neurons on other planets! primordial soup in data centers! wait…). The only way progress has ever been made has been by a) learning facts and b) arguing about what they mean. In 1970, no one had ever seen a planet around another star. Sagan was just guessing when he said “I would guess it is somewhere around one. In our solar system it is certainly several.” There’s so much we haven’t learned yet today, and I’m talking about both aliens and AI. Progress looks like fuzzy numbers until it doesn’t.
The first weekend of October, I’ll be at The Curve in Berkeley. There is an increasing-but-still-minuscule amount of attention paid to how the coming AI transformation will affect people living in the poorest countries. I’m continuing to work on some related topics within Google DeepMind, and will also have some blog posts out on the topic this month.
I’ll be in Sierra Leone from the 5th to the 19th. It’s a long shot, but if you or anyone you know is in Freetown, please reach out! Afterwards, I’ll be in London and Oxford for a week or two, and I’m excited to catch up with all the friends there. At the end of the month, I turn 30, about which I’m largely ambivalent.
Pontius Pilate
Ann Wroe’s Pontius Pilate (1999)
Martin Goodman’s Josephus’s The Jewish War: A Biography (2019)
Diarmaid MacCulloch’s A History of Christianity: The First Three Thousand Years (2009)
Pontius Pilate (1999), a biography by Ann Wroe, was my favorite book of the month, possibly the year. Wroe has been the obituaries editor for The Economist since 2003, filling the back page every week, and some of them are works of art. She wrote Pilate a few years before she took that job, while she was the paper’s US editor, and it builds a whole life out of almost nothing; what do we truly know about Pilate? The British edition is subtitled The Biography of an Invented Man.
There was almost nothing to go on. We do not even know his praenomen, the name his mother and wife and friends called him by. The only physical evidence we have of this man is one inscribed stone and a few small coins.
What else should we trust about Pilate besides this stone, besides these coins? There are a few pages of Philo and Josephus, and the Gospels. From these Wroe gives Pilate a Samnite family, a career in the army, afternoons at the baths, and a barber every morning. Next to that speculation she covers two thousand years of Pilate’s other appearances: apocryphal gospels, medieval plays, paintings, novels and films.
We know Pilate because of the trial of Jesus. Tolstoy was obsessed with this scene, as was Bulgakov and perhaps Monty Python. Wroe’s coverage of the trial is lovely. She goes through the four Gospels slowly and it is impossible to settle on a single Pilate. Did the trial matter to him at all? There is no certainty. Would he have remembered it the next morning, or brooded over it for the rest of his life?
It was only another day in his life. It was just a bad morning, like any other bad morning, in a man’s long struggle to keep to some sort of moral path. … And then he went to lunch.
Two decades later, in a podcast, Wroe said again: “And then he went to lunch. Because, you know, it’s just another day, another mad Jewish guy.” MacCulloch, in his scholarly History of Christianity, agrees. He says Jesus’s death “seemed at first a matter of little consequence amid the great affairs of the empire.”
Josephus’s history of the Jewish revolt against Rome in 66 to 70 culminates with Jerusalem taken and the Temple burned. Josephus had been a rebel general in Galilee early in the war, and had surrendered to the Romans when all his compatriots committed suicide. He wrote the book in Rome as a client of the emperors who had crushed the revolt. Goodman’s book is a biography of the War itself: how Josephus came to write it, and how it has been received over the past two thousand years. There is a lot of extraneous detail, and I couldn’t tell whether it was written for specialists or for the public. But all the good material is in there, including the translated highlights from the War at the back, if you’re not up for the thirteen hundred page original.
Josephus wrote about a man named Jesus. He wrote about a single man named Jesus, Jesus son of Ananias, “an ordinary farmworker,” who came to Jerusalem in 62 for the festival of Tabernacles and started crying out in the Temple: “A voice from the east, a voice from the west, a voice from the four winds, a voice against Jerusalem and the temple.” Some of the leading citizens had him beaten. When that didn’t stop him, the magistrates brought him to the Roman governor, Albinus. Like Pilate, Albinus had his Jesus whipped.
There, though the whips were lacerating him to the bone, he made no plea for mercy and shed no tear, but with his voice pitched in the most sorrowful tone responded to each lash with ‘Woe to Jerusalem!’ When Albinus, the procurator, asked him who he was and where he came from, and why he was broadcasting this message, the man made no reply to any of his questions, but just kept on reiterating his lament for the city. In the end Albinus concluded that he was mad, and released him.
The charge was different (Jesus of Nazareth was crucified, perhaps, as a would-be king of the Jews), but the hearing is very close to the one before Pilate about thirty years earlier. There is the beating, the silence, and the question of where he came from, which John’s Pilate also asks. Wroe returns repeatedly to the possibility that Pilate thought Jesus was mad. She mentions Jesus son of Ananias, too, at the moment Jesus of Nazareth is brought in to Pilate, as an example of priests bringing a religious case to the governor and nothing more. Albinus let his Jesus go, and Jesus son of Ananias kept crying woe for seven years and five months, never spoke to anyone, gathered no followers, and was killed by a stone from a Roman catapult during the siege which brought woe to Jerusalem.In 1959 the son of the Hebrew writer Micha Josef Berdyczewski published his late father’s argument that Jesus son of Ananias was the real Jesus of the Gospels. It failed to persuade scholars. Among other problems, Josephus is adamant that this Jesus was not executed.
For Albinus, that hearing was just another day, and we only know about it because Josephus wrote it down, as he wrote so much down. It is easy to imagine the hearing in front of Pilate may not have mattered either. Whatever made the difference between the two Jesuses happened afterward.
What happened after? MacCulloch is helpful, with plenty of side quests that weren’t relevant to me. Of the three thousand years in his subtitle, I was interested in 300 at most. Many of these years are empty. MacCulloch also thinks the crucifixion alone would not have made a new religion:
If that lingering and humiliating death on the Cross had been the end of the story, then the tale of Jesus would have remained embedded in Judaism. Jesus might have made it into the history books, even inspired a new departure in Jewish faith, but there would have been little likelihood of a separated or wider religion.
What came after the crucifixion is invisible to us. Up to the end of the first century, MacCulloch writes, we see the churches almost only through the New Testament, and that “there is a silence of about six crucial decades.” The worst of it is the fifteen or twenty years between the crucifixion and Paul’s first letters. How can we know who led the first followers, how they worshipped, or how the movement spread, or who founded the church in Rome, which was already strong before Paul arrived? At the start, MacCulloch says, the movement was “too insignificant to leave artefacts or even much trace in literary sources outside those which Christians themselves created.”
As Christianity disappeared with Christ in 30, so did Pilate. He was governor until the end of 36, so the first years of the new movement happened on his watch. Did he ever see James, Jesus’ brother? Wroe imagines him in Caesarea while Saul, the Roman Paulus, passes through on his way to Tarsus.
There his days unfolded as usual: the morning shave, the afternoon bath, the letters and briefings, the dinner parties. … The future St. Paul merely walked under his windows, went down to the harbor, inched across the blue sea, while the governor worried about something else.
When the new sect reached Caesarea itself, the home of Pilate’s palace, “Pilate did not seem to notice that either.”
Josephus was born in Jerusalem in 37, the year Pilate was summoned to Rome, and grew up there in the 40s and 50s while James led the church in the same city. His War says very little about these decades. Goodman calls its account of the first thirty years of direct Roman rule in Judaea, which include Pilate’s ten years, “very skimpy,” and notes that “the gap is papered over by an extended description of the Essenes”, of whom MacCulloch also has a peculiar obsession. The War never mentions the Christians.A short passage about Jesus appears in his later Antiquities, though Goodman says it “has been corrupted by Christian additions.” The medieval Slavonic translation of the War, which Wroe quotes a few times, has passages about Jesus and John the Baptist that are not in the Greek, and Goodman thinks these additions are Christian.
How did Christianity avoid notice and record? It was a time of religious chaos, with new churches and sects and splinters, and it would have been very hard to predict which of them would last. MacCulloch counts “at least four identities for Judaism, Sadducees, Pharisees, Essenes, Zealots, and probably many lesser sects,” and says the Christians at first “seemed just another minority answer to the problem of Jewish identity.” Josephus thought the sect that mattered was the one founded by Judas the Galilean, who led the resistance to the Roman census in the year 6, blaming his teachings that Jews should obey no master but God for the war itself. The prophet who ended Pilate’s career was certainly not Jesus, as medieval legend tells it. It was a Samaritan who promised to show crowds the sacred vessels Moses had buried on Mount Gerizim. Pilate’s troops attacked the rabble rouser, the Samaritans complained to the governor of Syria, and Pilate was sent back to Rome. Wroe:
Fanatical Christian statements did not seem to get reported in Caesarea, but this, in some garbled version, did.
Wroe assigns the Samaritan the identity of Simon of the Acts of the Apostles, a magician whom the apostles baptized and Peter sent away. In MacCulloch, Simon Magus turns up again later, as one of the men that gnostic belief systems were attributed to.
When did it become inevitable that this would be a Christian world? Probably not at the hearing, or the crucifixion; neither were events of significance. It could have been something in the missing years that we will never see. It could have been the destruction of Jerusalem in 70, which began the end of the Jewish Christian church in Jerusalem and left room for Rome to grow, though MacCulloch writes that “no one would have realized this even two centuries after the death of Jesus Christ.” It could have been Constantine. “In 303, as persecution of Christians gathered momentum in the empire, the last thing anyone would have expected was for the Church to enter an alliance with the Roman state.” Constantine backed the Christians when they were still “a decided minority in the empire.” Unlike Pilate, we have masses of information, portraits, accounts of Constantine, but we are no closer to understanding him. Why did he do what he did? What did he think of Christ, of his god? MacCulloch thinks his motive “will probably remain hidden from us.”
Whenever the turn came, the missing years had left room for later people to reinvent Pilate and Josephus. Pilate’s reinvention started in the Gospels, the earliest records we have of him. MacCulloch:
Most Christians did not want to be enemies of the Roman Empire and they soon sought to play down the role of the Romans in the story. So the Passion narratives shifted the blame on to the Jewish authorities, and the local representative of Roman authority … was portrayed as inquisitive and bewildered.
Early Christians needed the empire to leave them alone. The most useful Pilate, to them, was one who said Jesus was innocent. Tertullian wrote that Pilate was “a Christian in his heart.” The Ethiopian church made him a saint, with a feast day on the 25th of June that he shares with his wife, who perhaps dreamed of Jesus the night before the trial. When Christianity became the religion of the empire, however, accepted and official, in Wroe’s words “Pilate’s fortunes fell into steep decline, and he became a villain in his own right.” He stayed in the Creed the whole time, “crucified under Pontius Pilate,” where Wroe calls him “a bureaucrat set down incongruously among elemental streams of light and celestial thrones.” MacCulloch thinks “it would have been better for the moral health of Christianity if the blame had stayed with Pilate.”
Josephus was also invented and reinvented. First, Christians kept his book alive because it seemed to confirm Jesus’s prophecy that the Temple would fall. In Goodman’s words, “for the rest of antiquity, the book had a life only within the Church.” Eusebius, the church historian who wrote under Constantine, cited Josephus that Jerusalem was destroyed as punishment for the crucifixion. Many manuscripts changed its title from The Jewish War to Conquest, and Basil of Caesarea said Josephus had written his history “for us.” We have no evidence the early rabbis read him at all. Josephus “would have been astonished and horrified at the reinterpretation of his narrative as a divine judgment on his people for rejection of Christ.”
Jews returned to Josephus in the modern era. Jews put him on trial. Law students in Antwerp found him guilty of treason in 1937. A schoolboy in the Vilna ghetto took part in a trial of Josephus in March 1943, while the Jewish council the Nazis had set up was trying to protect the ghetto. The French Resistance condemned him for collaboration. Mock trials became a staple of Zionist youth groups, and a trial on Israeli television in 1992 acquitted him for lack of evidence. The trials Goodman reviews are crazy, and there are so many of them! American high schools and TV channels don’t hold mock trials of Benedict Arnold. Goodman finds it “rather bizarre” that Josephus has so often been judged a traitor, since he spent his last decades in Rome writing at length and with passion about Judaism, which the regime despised. It’s hard to call someone a traitor two thousand years later. The poet Yitzhak Shalev ended a poem about him:
Year after year he stands trial before me,
And my hand, reaching out to seal his fate,
Is suddenly stilled.
The verdict has been suspended
To a later date . . .
As for Pilate himself, he dropped out of the record early, and all we have of him are inventions and reinventions. He probably left Judaea in December 36 and went home overland, two thousand miles through Anatolia and Greece in winter. Wroe invents felt overshoes to keep him warm. He reached Rome to find Tiberius, the emperor who appointed and condemned him, dead. “Pilate disappears from the ancient accounts. Josephus follows him no further.” The Christians could not give him the same peace.
Pilate would have to remember Jesus, the early Fathers fretted; he could not do otherwise. Somehow, he would have to pay the price.
Eusebius gave him a suicide (“apparently, divine justice didn’t spare him long”), and later legend had him wandering between Lausanne and Vienne and tied his name to mountains and pools and bad weather all over Europe. Behind all of it, Wroe says, “lay the medieval conviction that Pilate could not stay still.”
I think he probably never thought about Jesus of Nazareth again. Rome was a big city, and the church there was small and Greek-speaking, and had probably grown out of the Jewish quarter across the Tiber. When Rome did notice the Christians, a decade or more after the crucifixion, Suetonius records that the emperor Claudius expelled the Jews for rioting “at the instigation of Chrestus,” which MacCulloch calls “probably a garbled reference to Christian preaching.”
My preferred elderly Pilate is Anatole France’s, from his 1892 story “The Procurator of Judaea.” France’s Pilate is exiled to Sicily, grows fat and gouty, and does his brooding about Judaea “over lunches of larks in honey in the shade of terraces of roses.” He remembers the slights of the Jews, the silk dresses and the dancing girls. An old friend asks if he remembers a young miracle-worker from Galilee.
Jesus? Jesus of Nazareth? I don’t remember him.
Links
Here is an actual, real, line from the initial script for Star Wars Episode II: Attack of the Clones:
My summer in London was very hot, and I was excited to escape before the August heatwaves. This is the most interesting thing I saw come out of the hot dry summer!
The Financial Times had the first Taliban-approved interview with Hamid Karzai in the past three years. If you haven’t been to his brother’s restaurant in Baltimore, The Helmand, I can recommend it!
Also, Karzai has an eight-year-old daughter and his wife is 56 years old and a gynecologist (I believe, the first female gynecologist in Afghanistan!). That is a remarkably geriatric pregnancy.
This is a hilarious account of neighborhood drama and public finance in the East Bay. If I ever get a Nextdoor account, confiscate my electronics, please.
After all, I have a 30-year mortgage. I’m not locked in here with him. He’s locked in here with me.
Here’s John Psmith on Inga Clendinnen’s Aztecs. The Aztecs didn’t last long. As Clendinnen puts it,
a man whose grandfather had fought as a hireling in another city’s wars in the Mexica’s early days in the valley would himself have seen the glory of their achieved empire, and his son watched its destruction.
Wow, the Aztec world was very unpleasant.
If the essential conservative insight is that things can always get worse, then it’s funny that conservatives often fail to imagine just how much worse things might get. The nine most frightening words in the English language might be “the distribution of possible human societies is very wide.”
Computers are Bad has a fun history of technologies to detect, prevent, and survive avalanches. It was the first I’d heard of meteor burst communications, and it’s fun to see that it’s not only AI companies finding a way to employ moral philosophers:
These enhanced capabilities of W-Link seem to have led to enough controversy that the Wikipedia article devotes much of its length to ethics discussions, and manufacturers have put out position papers on their decisions to implement or not implement certain W-Link features. The concern is that W-Link may give rescuers more information on which to prioritize rescue in a discriminatory way, perhaps by searching for victims with (more expensive) W-Link beacons before less expensive conventional beacons, or by using knowledge of W-Link serial numbers to prioritize the search for specific people.
A somewhat-rigorous proposal for geoengineering to cool the earth. I think of termination shock like nuclear semiotics: fun thought experiments that, if we get to the point where they’re relevant, thank goodness! Things have gone right.
I was watching the viral mom at the Giants game moment live (if you don’t know about this, you should skip it). The post by the mom is mostly AI generated but I’m super ok with that; the odds that something like this would happen to someone who could write well are super low, so this is the best case for AI writing: democratizing communication.
Previously
…