See also: The Irony of “Progress Studies”

Defined by Will MacAskill for EA Forum, Longtermist is “the view that the most important determinant of the value of our actions today is how those actions affect the very long-run future.”

The irony is that this only holds true in the abstract. According to an Open Philanthropy estimate and AI Expert Surveys, there’s a 50% chance of transformative Artificial Intelligence emerging by around 2050. If that happens, basically nothing we do in the meantime will matter, at least not with regards to total expected utility. Ensuring that the AI is safe, human-aligned, benevolent, etc, is of primary and nearly sole importance.

If you take this idea seriously, we should be obsessed with the short term to the exclusion of all other timescales.

This has practical implications. For example: should you expend energy cultivating the next generation of scientists, or just focus on your own research output? If we seriously only have 30 years, the latter becomes much more compelling.

Similarly, if you’re serious about longtermism, the altruistic case for having children becomes much weaker. Particularly precocious offspring might be able to productively do longtermism-relevant work in their mid-twenties, but that’s not enough time to recuperate the cost of raising them. [1][2]

The average age of respondents to EA Survey 2019 was 31. Similarly, SSC’s Survey gives a median reader age of 30. So even if it’s a bit over 30 years to transformative AI, it’s not as if the existing cohort of longtermists will die off. Will MacAskill will be just 63 in 2050, Hilary Greaves 71 and Toby Ord 70. Nick Bostrom will be the oldest at 77, but he reportedly skips meals to drink a vegetable “elixir”, so I think he’ll be okay.

Quick step back: I’ve been duplicitous in my use of “concern”. The longtermist mindset is something like: “We care immensely about the long-term, that’s why we focus immense on the short-term”. But that’s precisely my point. This isn’t a deep contradiction, it’s not hypocritical. It’s just ironic.

Score another point for nominative anti-determinism.

Footnotes
[1] If you’re really altruistic (and perhaps sociopathic), you could have kids who you never see, though perhaps children treated that way are unlikely to follow in your footsteps, and may actually be perversely likely to become some kind of scorned longtermism supervillain.

[2] This also means Alexey Guzey’s offhand criticism doesn’t land with any particular force.

As usual, you could have skipped this entire post and just read a tweet instead.

Since I first read Stubborn Attachments in 2018, I’ve struggled with its foundational provocations. Has growth been good? Will it continue to be in the future? What if it kills us all first? Today, I’m finally prepared to share an article aimed at answering these questions.

The Moral Foundations of Progress is available here as a pdf.

This is the culmination of my thoughts on Progress Studies as an intellectual movement, and also on the budding institutions that surrounds it. Though I’ve previously written abstractly about the need for foundational assumptions, in this new piece, I actually argue for them.

In other words, don’t worry too much about “field-building”. Simply embark on a quest to rigorously answer important questions, and invite others to join in the adventure.

With that in mind, I hope my article will inspire work far beyond its relatively narrow scope. Work not only in research and writing, but in institution building, community organizing and movement towards applications.

If you would like to get involved in any capacity, you can find my contact details here. I look forward to hearing from you.

Thanks to Leopold Aschenbrenner, Basil Halperin, Alvaro de Menard, and Philip Trammell for their comments on an earlier draft.

As a rule, autobiographies are horrid.

People should not be trusted with writing their own legacy. Certainly, not after they’ve aged into has-beens reminiscing on the glories of their youth. [1]

Jordan Mechner’s The Making of Prince of Persia avoids this trap by being written in real time. It is not a polished accounting, but merely his diary from the time he spent working on Prince of Persia, starting with his college graduation, and concluding 4 years later.

It’s also flat out the best biography I’ve ever read. I started at 10pm last night, stayed up until 2am to finish it, then until 4am to play Prince of Persia on an emulator, went to bed, woke up, and then immediately read the book a second time to write this review.

Least I sound like a rube for lionizing a video game designer’s diary, I promise I have read the “good” biographies too. Under lockdown alone, I’ve gotten through The Power Broker (Pulitzer), Isaacson on DaVinci (#1 NYT) and American Prometheus (Pulitzer). With apologies to Kevin Kwok, they are all drastically inferior.

Mechner begins the saga as an unlikely hero. He already made a best selling video game in college, but is now filled with doubt, living with his parents, and trying to write a screenplay. He writes incessantly about how the games business is “dying up”. How the Apple II is a “dying format”. How “nobody knows how long the games market will be around”. He’s like a self-destructive Jeff Bezos, fully prescient, but choosing to pursue the worst possible market at the worst possible time. The fact that Mechner turns out to be deeply wrong about all this does nothing to diminish the death knell-quality of this whole project. It’s not a career. It’s a dying shriek fading into darkness.

It’s not so much romantic as absurd. In the course of making Prince of Persia, Mechner:

• Takes 6 months off to write a screenplay.
• Drives out to Skywalker Ranch, meets George Lucas, fails to get his script acquired.
• Applies to NYU film school, gets rejected.

It would be crazy enough if this was a backwater origin story, but it’s happening in parallel with all of his greatest technical accomplishments. It’s like if Brian Armstrong took a year off from founding Coinbase to pursue acting, while Coinbase was already taking off.

Even once the game finally ships, Mechner worries it will flop, and continues to mope around. A full year after initial release, Mechner is at NYU. He’s already been rejected, but he’s hoping he can hang out around film students and learn something anyway. So it’s September 1990 and he’s playing gofer on a student film set fetching coffee. Meanwhile, Prince of Persia will go on to sell 2 million units and receive praise as “the first cinematic platformer” and “one of the greatest video games of all time”.

The whole time, Mechner is aware that he’s strangling the golden goose. This is how he operates:

I have no excuse for slacking off. As Adam Derman once told me in a letter (about Karateka): “You dumb shit. You’ve dug your way deep into an active gold mine and are holding off from digging the last two feet because you’re too dumb to appreciate what you’ve got and too lazy to finish what you’ve started.

There could be a certain delight in watching Mechner stumble about. We might, as readers, indulge in the secret knowledge that he will succeed, and take a kind of perverse joy in knowing that all his struggles will pay off in the end. But the weird thing is, Mechner already seems to be experiencing that same effect in real time. In some episodes, he doesn’t take his game seriously and flies off to LA for weeks to meet with agents. And then he’ll come back and say “this will be one of the greatest video games of all time”. Or a few pages later: “This is going to be the greatest game of all time.” And later: “There’s no other game that even remotely approaches this.”

It’s not quite a mantra, but it’s not arrogance either. He’s just manic-depressive. As he writes only shortly afterwards: “Have I ever had what it takes? Am I losing it? Give me a signal; show me a sign. Where’s the meaning in all this? Nobody cares about the fucking game, not even me. Why am I doing this?”

In the past couple months, I’ve been writing about innovation from an organizational level, asking what kinds of structures allow breakthroughs to happen. The Making of Prince of Persia casts serious doubt on that entire agenda. It doesn’t seem to matter one bit what structures exist around Mechner. He is a force of nature. A host unto himself. As long as he has his Apple II, a surge suppressor and his notebook, he can design this game.

The book is deeply solipsistic, as most diaries are. When other people are mentioned, it’s mostly because they’re getting in the way: “It’s not that I insist on doing everything my own way. I’m always hoping someone else will come up with something better than I would have done myself. But when they don’t…?”

He does work at an office, and there are people who technically employ him, but they’re in the background, irrelevant if not annoying. Eventually he has to hire more programmers to work on ports for other systems. Some of them are competent, others less so: “Jim’s work is dismayingly bad. I’m not sure he’s saved us any time at all. I’ve got no choice but to redo it.”

At one point Mechner is assigned a new product manager, and seems to like him: “The meeting erased any doubts I might have had about Brian’s effectiveness as a product manager. This is what I needed all along: someone to push me.” This is fine, but Brian’s role in the rest of the book seems limited to helping Mechner avoid contact with other people: “Another nightmare is the box copy. The art department’s new draft sucks. I asked Brian if he could throw a fit and insist they use my draft? He seemed willing to try.” and “Prince has no better champion than Brian. He’s been fighting for a year. He’s powerless, that’s all.”

I don’t know if innovation tends to happen in isolation, but Mechner certainly makes the case that it can.

The other horrid aspect of autobiographies is the vicariousness of it all. You read about someone’s success. You hear about their life. You’re never really there.

The Making of Prince of Persia avoids this trap too. It’s not that we get unusually close to the action, so much as Mechner gets unusually far away from himself. As he writes on June 13, 1989, a few months before the release of Prince of Persia:

Everyone has their own particular form of self-destruction. Mine, I’m starting to think, is standing outside myself, watching myself live my life, turning my face so as to give the cameras a better angle, and thus missing the whole thing.

The book, again, is fantastic. I read it on Kindle, but the physical Stripe Press copy has nice illustrations. You can watch some behind the scenes shots on Youtube, read the source code on Github, or play the game online in MS DOS emulators.

Jordan Mechner, 1989. From the author’s website.

Endnotes
[1] Richard Feynman is honored for his Nobel Prize in Physics, but he’s famous for Surely You’re Joking, Mr. Feynman! As fellow Nobel winner Murray Gell-Mann repeatedly grumbled, the stories were only interesting because Feynman “spent a huge amount of energy generating anecdotes about himself.”