After arguing that industrial research labs won’t return, I was hopeful for a new mechanism to reignite transformative research, but had no idea what it would actually be.

A couple hours later, Nintil told me about Focused Research Organizations.

In short: FROs promise to pair government funding with startup agility, and a mandate to pursue high-impact pre-commercial research. There isn’t a lot of literature out on them yet, but it’s a compelling proposal. [0]

Going into this, it’s worth noting that Adam explicitly says FROs are not a replacement for existing institutions, but they might be good at the margin. Specifically, the proposed budget is $1 billion over 5-7 years, around 0.3% of all federal R&D funding.

As with everything, I have a lot of criticism, but let me start by saying that I’m excited! We need new ideas, and actually experimenting with funding models will help us advance much faster than musing in a vacuum. The existing systems are filled with their own flaws, and I’m not attempting a cost-benefit analysis, so nothing critical I say should be taken to mean that I am net negative.

But first, what does Adam actually envision?

Summary of the FRO Proposal

The bulk of Adam’s thinking is laid out in the FRO Whitepaper. There’s also some discussion in the Idea Machines Podcast, and a bit more in a talk he gave with Nintil a couple months ago.

The core proposal: “Startup-like organizations, but pursuing pure science outcomes with no market.” In another venue, Adam describes them as “a special purpose organization to pursue a defined problem over a finite period of time.”

Or expressed negatively: FROs target projects that cannot be addressed by any existing organization or funding mechanism. The university system is massive, receiving around $40 billion in federal funding each year. Last time, I argued that this was one of the principle reasons behind the death and continued absence of industrial research labs. Why pay to innovate in-house when there’s a $40 billion in research happening for free?

Reasoning backwards from other people’s first principles is precisely the way to get around this. Adam understands the university system deeply, knows what it’s mechanisms cannot produce, and aims to directly target the research that can’t currently happen.

You can think about this through analogy to startups. A small team of 2 founders with $125k in funding could never reasonably compete with Google head-on. The first step for many pitch decks is to explain why, if the idea is actually good, it hasn’t already been done. [1]

So where do universities and corporate labs systematically fail? Adam argues it’s projects which “require levels of coordinated engineering or system-building inaccessible to academia” and “benefit society broadly in ways that industry cannot rapidly monetize”. Stated elsewhere, he says the fundamental tension is between academic settings which incentivize individual achievement over team coordination, and industry organizations which are better at teamwork, but don’t aim to produce public goods.

It’s worth being really clear about what this last claim requires. It’s not that Adam is condemning startups or capitalism, or that he believes industry never produces public goods. The argument is merely that there are some projects, at the margin, which:

1. Require teamwork and cross-disciplinarity
2. Only make sense if you intrinsically value the resulting public good

In this sense, rather than the abstract basic/translational/applied trichotomy, Adam talks about projects that are “pre-commercial” or don’t have a clear path to monetization in the short-term. [2]

That’s the narrative about why FROs ought to exist in the presence of existing organizations. But how do they actually work?

The federal government (or philanthropist) commits $1 billion over 5-7 years, paid out to 16 projects, each with a focused research charter. These ought to be time-bound, and there should be a clearly defined end goal that results if the project is successful. The whitepaper also notes that each should be led top-down by a CEO in the style of startups, rather than by committee or by a decentralized collective of loosely affiliated researchers.

The initial line sums it up pretty well. An FROs is a government-funded organization, run like a startup, focused on pre-comercial science, operating over a finite time horizon.

Commentary

Startups are great, but we should understand them ecologically: the result of a precarious balance held in place by a surrounding ecosystem. Swap out one part, and you don’t get “startups but for X”, you get a hot mess. [3]

Adam proposes taking startups, and making the following modifications:

• Instead of a profit motive, FROs follow a scientific charter
• Instead of pivoting as they run the idea maze, FROs have a specific predetermined goal
• Instead of market signals and user validation, work is guided top-down by a CEO
• Instead of VCs who compete for opportunities and bet against each other, the only funding agent is the federal government

I worry that any one of these substitutions would be fatal, and it’s not clear that they collectively bring us to a new stable equilibrium.

While there are some legends of startups that worked hermetically for a year or two, it’s hard to think of tech companies that were pre-commercial for 5-7 years, and maintained an attachment to reality. The famous anecdotes all turned out to be vaporware or massively over-hyped (Theranos, Magic Leap).

Another concern is with leadership. Startups are able to move quickly by vesting their CEO with dictatorial power [4], but they’re ultimately beholden to metrics. There is centralized power and a cult of personality, but their authority still stems from each employee’s faith in growth, validated against a steady exponential growth in valuation, revenue and/or user base.

It’s possible FROs could supplant the entire process with a rigorously predefined scientific charter. In one podcast, Adam mentions literally using “characters sequenced” as a measure of progress for the human genome project. The whitepaper states that in general, “FROs should [be] driven by quantitative metrics and/or concrete design goals”.

This also addresses a key concern over any transformative research: measuring impact. If a result is novel, there’s no point of comparison by definition, and no way of justifying the costs relative to an expected baseline. Even if the work does end up being impactful, it might be 20 years down the line. Running the entire program around a focused charter and quantifiable outputs alleviates some of these concerns. We still won’t know if the chosen projects were the right one, but we’ll at least find out of the program succeeded on its own merits.

If executed correctly, this would provide accountability for the CEO as well, in the same way that growth metrics provide accountability to founders. With a sufficiently well-defined goal, there’s little ambiguity over how well the company is being run.

Finally, I worry about risk. Though the Startup Ecosystem works perfectly well, we have to distinguish between the broader trends and individual firms. At the micro level, each startup is dysfunctional and overwhelmingly likely to fail. Part of this stems from market dynamics, but it’s also the nature of a scrappy ambitious project with a centralized authority and thus a single-point of failure. The exact failure rate depends on your reference population, but a commonly cited estimate is 90%.

If this holds true for FROs, a 16 project portfolio would still have a 19% chance of total failure. Trial and error is the nature of scientific discovery, but the greater harm might be a vast chilling effect on future experimental models.

Doubling the scale of FROs to 32 would get us down to a 3% chance of total failure, and going up 10x to 160 projects would get us down to a mere 0.0000047%.

The expected value doesn’t change, but that doesn’t matter if like startups, FROs are a hits-based enterprise. In venture capital, one big win pays for the entire portfolio. In FROs, one human genome project [5] or [connectome](https://en.wikipedia.org/wiki/Connectome#:~:text=A connectome (%2Fkəˈ,an organism’s nervous system.) could justify the entire budget.

Even then, the whole proposal would amount to just 3% of federal R&D, and only for the brief 5-7 year trial period. [6]

In summary: FROs are promising, and the program should be 10 times bigger. [7]

---

Thanks to Adam Marblestone to reviewing a draft of this post.

For the original proposal, see his whitepaper with Sam Rodriques.

---

Footnotes
[0] Adam mentions over email that these could also be philanthropy-funded. In the Idea Machines podcast, he and Ben express some concern over salary-restrictions in government-run projects. Taking Patrick Collison’s notes on the importance of compensation, it’s worth wondering if FROs are even possible with government funding.

Having said that, ARPA-E does give grants to private companies who (I assume) can set their own salaries. I’m not sure exactly where the line is, or if the distinction is between government-funded versus government-run, or if there’s any room for exceptions.

[1] Learning where tech giants systematically fail is an underappreciated reason to get a corporate job for a few years before setting out on your own.

[2] The short-termism of Venture Capital is overstated. You can find an arbitrary number of companies passed up for investment because they lacked a path to monetization, but there are compelling counter examples that suggest VCs are willing to make long term bets so long as the payoff is sufficiently large. Consider Magic Leap, which was founded in 2010 and didn’t have a commercial product until 2018. It’s now considered a failure, but that makes an even stronger case of VC long-termism. They’re willing to fund speculative, long-term, unproven pre-commercial technologies, even when (empirically) there is significant risk of failure.

How does this happen? One answer is that VCs are not actually long-termist, they just get tricked into making long-term decisions. Perhaps Magic Leap told investors in 2010 that they would launch in 2015, then just kept moving the goalposts, and took advantage of sunk cost. Or they just raised from different investors, claiming each year to be under 5 years away from commercialization.

A priori, is there any reason to think VCs even have a short-time horizon? Of course they want returns eventually, but the short-term goal is to have a strong enough track record to raise another fund. That might mean having your portfolio companies hit commercial milestones, but it could just as easily mean that they were able to raise money from other VCs at increasingly high valuations.

[3] Despite the frequent comparisons to startups, the whitepaper makes it seem like FROs are actually much more like National Labs. In a 13 point comparison table, FROs differ substantially from everything else, but have only two points of contrast with National Labs:

• “Exists as an autonomous organization mobilized in a rapid agile fashion”
• “Provides strong support for post-project transition to commercialization”
• FROs aren’t permanent, and don’t provide a clear career path

It’s worth taking a minute to understand what the National Labs are. There are 17, each administered by a different entity, some embedded in universities, others by industry. It’s all under the Department of Energy, but the majority of their funding (55%) is for weapons research, mostly around the US nuclear arsenal. Overall, it’s $12 billion total in annual budget, which is two orders of magnitude larger than the total proposed FRO budget.

It’s not really accurate to say that an FRO is a National Lab, but more agile and with support for post-project commercialization. It’s more like a centi-lab with startup characteristics.

With that in mind, it feels like an easier way to pitch this whole proposal would be: National Labs, but for the life sciences, and as a small proof of concept.

[4] Abuses exist, but are largely moderated by at-will agreements and an active market for startup employees.

[5] Adam brings up the Human Genome Project a few times as an example of a past project that could have been successful as an FRO, but the whole line of augment is a bit confusing. FROs are supposedly for funding projects that couldn’t happen otherwise. Suggesting that they might have funded something that already did historically happen seems to weaken this argument.

[6] This is factually accurate, but it’s not a great line of reasoning. Lots of things would be “only x%” of some federal budget line item. It’s easy to say “The US government could invest $20B into climate justice, for just 0.1% of it’s total budget!”

[7] To be clear: each organization should still be the size Adam proposes ($25-$75 million over 5-7 years).

Nintil suggests: “Maybe there was a monocausal event in 1970.”

Many graphs to follow, but first:

• The Mother of All Demos: 1968
• Intel Founded: 1968
• Moon Landing: 1969
• Woodstock: 1969
• First ARPANET Connections: 1969
• First Commercial Video Game: 1972
• Roe v Wade: 1973
• First Cell Phone Call: 1973

And of course, WTF Happened in 1971.

Constitutional Amendments [1]

Energy Use

GDP Doubling Time

Peer Review

Donald Braben, author of Scientific Freedom: “'academic research before about 1970 was essentially unmanaged”

Leaded Gas

Oil Shock

Inflation

Male Income

R&D’s Share of the Federal Budget

Senate Filibusters

See Also:
Scott Alexander: Wage Stagnation: Much More Than You Wanted To Know
Scott Alexander: 1960: The Year The Singularity Was Cancelled
Noah Smith: How the 1970s Changed the U.S. Economy

Footnotes
[1] I’m cheating a bit here. The 27th amendment was proposed in 1789, but ratified in 1992. So we have ratified something since 1971, we just haven’t ratified anything written since then. Having said that, it’s a really boring amendment.

Interpretation
Okay, the obvious objection here is that it’s easy to cherry pick examples for any 5 year period. But is it actually? Maybe you could do WWII, but that wouldn’t be weird, that would just be WWII. This is notably because the events seem largely unrelated.

But you might be right, and it’s possible this is all just noise. I’m genuinely unsure.

A compelling explanation would be that a lot of this is related. Energy use slowed becaused GDP per capital slowed. Inflation skyrocketed because we got off the gold standard. We stopped passing constitutional amendments because of the filibusters.

Compiling this post, I came across several mentions of Mike Mansfield. He’s responsible for changing the filibuster bylaws (1970), prohibiting military funding of research without a direct military application (1969) and limiting DARPA’s scope (1973). Darpa, of course, was responsible for both ARPANET and the Mother of All Demos. For more on why this matters, see Steve Blank’s Secret History of Silicon Valley, summarizing the influence of military funding on seemingly unrelated innovations.

So sure, there are some specific causes we can point to.

But just imagine being around in these years. In 1962 JFK announces we’re going to the Moon, and a mere 7 years later we’re there. That very same year, the internet comes out. 3 years later we have video games, and a year after that, cell phones.

Of course, the average American wouldn’t really have cared. The internet existed in some lab, not in your home. Maybe it’s the same situation today. We have CRISPR, but it hasn’t yet had a big impact on our lives. Google announced quantum supremacy, but I won’t have a quantum computer for a long time. GPT-3 can synthesize shockingly good music, but nothing I would actually listen to. It’s possible 50 years from now we’ll look back at 2018-2023 as an incredible period of innovation, with nearly miraculous coincidence.

Maybe.

---

[Edit: 2021/02/19] From Tyler Cowen:

The break point in America is exactly 1973 and we don’t know why this is the case. It’s often argued that 1973 is the breakpoint because the price of oil goes up a good deal because of OPEC and the embargo, that might be true. But since that time, the price of oil in real terms has fallen a great deal and productivity has not bounded back. But at least in the short term, that seems to be the relevant shock.

Bell Labs is dead. It remains dead. And we have killed it.

In fact, even the concept seems to be dead. The top result in Google for Industrial Research Lab is a retrospective history. Pluralize the query as Industrial Research Labs and the top result is a 1946 index of the many labs that used to exist. Try Corporate Research Lab and you’ll get the cheery article The Death Of Corporate Research Labs.

Summarized in this last piece, Rosenthal determines the cause of death:

Lack of anti-trust enforcement, pervasive short-termism, driven by Wall Street’s focus on quarterly results, and management’s focus on manipulating the stock price to maximize the value of their options

Looking backwards, this is a tragedy. How shall we comfort ourselves, the murderers of all murderers?

But looking forward, it’s an immense opportunity! So long as we remedy the underlying causes, the great industrial lab may rise again. We may already be on the right track. Antitrust pressure is rising, and privately held companies not subject to Wall Street’s demands are booming.

What about short-termism? Via Tyler Cowen, Warren (2014) finds “no clear evidence of flawed short-term oriented management practices”.

This is good news! The industrial research lab has fallen, but with the causes gone, it will rise again.

As Ben Southwood concludes his piece for Works in Progress “we will see the return of various large in-house labs.” [1]

---

I am not so optimistic. Regardless of the antitrust situation, industrial research labs will not return.

Southwood and Rosenthal’s findings are both derived from Ashish Arora, Sharon Belenzon, et al.'s The Changing Structure of American Innovation, worth reading in full. As he concludes:

It seems unlikely that corporate research will rediscover its glory days… For some time, quick wins from low-hanging fruit (such as optimizing auction or advertising formats) may cover up the problem, but the fundamental challenge of managing long-run research inside a for-profit corporation remains a formidable one… incumbent firms continue to rely on outside inventions to fuel their growth. In the longer run, therefore, university research will remain the principal source of new ideas for such inventions. [emphasis mine]

In other words: why bother innovating when you can let someone else do it for you? Firms will still engage in the development half of R&D, but this will take the form of translating existing findings into products, rather than breakthrough fundamental research. Arora & Belenzon again:

In summary, the new innovation ecosystem exhibits a deepening division of labor between universities that specialize in basic research, small start-ups converting promising new findings into inventions, and larger, more established firms specializing in product development and commercialization (Arora and Gambardella, 1994). Indeed, in a survey of over 6,000 manufacturing- and service-sector firms in the U.S., Arora et al. (2016) find that 49 percent of the innovating firms between 2007 and 2009 reported that their most important new product originated from an external source.

…As a result, federal research dollars for the university sector grew from an estimated level of $420 million (1982 dollars) in 1935-1936 to more than $2 billion (1982 dollars) in 1960 and $8.5 billion in 1985. Between 1960 and 1985, the share of university research of GNP grew almost twofold from 0.13 to 0.25… Corporate labs historically operated in an environment where university research and start-up inventions were scarce.

Arora & Belenzon cites a 4.25x increase from 1960 to 1985. What about the years since? Courtesy of the NSF:

Federal funding has continued to increase rapidly, up another 3.5x since 1985. For its part, institutional funding (meaning internal funding in the form of endowments, gifts and so on) has grown over 6x. Faced with this bounty, it’s no wonder firms lost their appetite for footing the bill themselves.

---

To some extent, this is great news! The industrial research lab may be dead, but that doesn’t mean innovation is over, it’s just coming from universities instead. More funding means more science can get done, and who cares if it happens in industry or academia?

Unfortunately, the research we see today is of a different nature. In a section titled “Inventions originating from large corporate labs are different”, Arora & Belenzon enumerate the kinds of innovations we’ve lost in the shift towards university labs:

• Corporate labs work on general purpose technologies
• Corporate labs solve practical problems
• Corporate labs are multi-disciplinary and have more resources

Again, the paper is worth consulting for full details, but it suffices to say that different mechanisms for attracting, nurturing, and managing talent will result in different types of outputs. None of this is to say that the corporate labs were on balance superior to today’s university labs, merely that we are missing out on some innovations (and likely getting others in return).

Though more sensationally, it’s worth worrying that there’s been a profound exodus of top talent from research into industry. As Patrick Collison describes:

One thing that I think is underemphasized is that [corporate research labs] competed on the basis of compensation. They just paid more than other potential sources of unemployment. PARC’s strategy for aggregating the best computer scientists in the world was to pay them more than they would be earning in academia. And in the 70s, you couldn’t Google and earn millions of dollars a year, Silicon Valley hadn’t really left the launchpad.

To a significant extent, the same thing applies to Bell Labs. They were quite explicit that their strategy was to compensate really well and present more favorable employment than academia… It could be the case that because there are so many high return loci for super talent people to go and deploy their talents, you could never quite aggregate talent to the same extent.

…it’s hard to think of any major successes from these kinds of labs over the last 10 or 15 years. [lightly paraphrased for clarity]

As the lamentation goes, “The best minds of my generation are thinking about how to make people click ads.” It’s hard to prove that there has been a brain drain, but if so, it would indicate a strict loss in quality, as opposed to the more stylistic shifts Arora & Belenzon describes.

---

Bell Labs is dead, and it’s not coming back. Since corporate labs are uniquely suited to some kinds of valuable research, this loss is troubling.

Where does that leave us?

One option is to attempt a massive overhaul of the entire system. If corporate labs have dwindled in the face of the government funded university system, we ought to redirect some portion of those tax dollars to industry instead and seek a better balance. To some extent, this already happens. DARPA awards funding to small businesses through it’s Small Business Innovation Research grants, as does the NIH.

Even here, corporate labs won’t have the same incentives they had in the glory days from 1940-1970. So long as the university system thrives, firms will pursue growth through innovations discovered externally.

In that light, I have a different proposal: Instead of reviving the corporate research lab out of nostalgia, we should consider the more specific goals these labs accomplished, and then target them directly. If we are eager for more multi-disciplinary or research more closely tied to practical problems, we ought to build institutions to pursue those particular aims.

Even more broadly, we can think of institutions as just one of many possible mechanisms to allocate human capital. Nadia’s Helium Grants are also a mechanism for talent allocation, as is venture capital, and as is Substack.

The scientific world is not merely a collection of modules that produce research in hermetic isolation. Rather, it is better understood as an interconnected ecosystem. Increased profitability in software may cause cost disease elsewhere. An exciting new topic in one field could cause a genius exodus in another. Without this understanding of how it all interacts, attempts to recreate a single piece of the 1960s without the supporting context are doomed from the start.

We should seek to understand conditions as they exist today, think deeply about the particular aims we wish to satisfy, then design improved mechanisms within a contemporary environment.

---

Thanks to Ashish Arora and Nintil for their comments.

---

See also
Nintil – Fund People not Projects
Alexey – Reviving Patronage and Revolutionary Industrial Research
Nadia Eghbal – Seed stage philanthropy

---

Footnotes
[1] Ben Southwood read the same paper as Rosenthal and came to the opposite conclusion: the decline was the result of too much antitrust enforcement. The two only appear correlated because success driven by corporate research leads to antitrust. His full quote is “Perhaps antitrust bodies will be restrained, and we will see the return of various large in-house labs.”, but my summary of the conclusion as optimistic still stands.

---

Frequently Asked Questions

What about Google Quantum?
I’m horrifically unqualified to make this judgement, but this does stand out as an important achivement.

Jack Hidary of Google’s quantum computing initiatives once said: “We literally created a spreadsheet of the experts in this space. We only came up with 800 names globally.” So maybe this is the exception that proves the rule, and Google was able to succeed precisely because there is not a thriving university system for quantum computing.

It’s also worth understanding the achievement as a result of Google’s partnership with NASA, though I don’t know the details of each party’s contributions.

What about AI?
Again, I’m not really confident here, but as I understand it, breakthroughs in AI consist largely of scaling up existing techniques, or inventing new techniques to enable greater scale.

If that’s a fair summary, the apparent dominance of firms in AI research would seem to be a product of their outsized resources, namely compute and data.

Still, why invest in research instead of applying the findings from academia? My guess is that the field is just moving too quickly, and being even a year or two ahead makes a huge difference.

Does this present a promising blueprint for other fields? Maybe. There are other fields that could see dramatic progress powered largely by advances in compute. Or maybe other fields that are amenable to AI-driven progress sooner than we expect. Though even in these cases, I would not expect Google to take over unless the results translate easily into profits. Instead, we’ll likely continue to see Google partner with universities, while staying focused on their own core competence.

What about Open AI?
OpenAI is a non-profit wrapped in a capped-profit LP, managed like a startup.

But okay, incorporation aside, why does it exist? It could be that AI Safety really is the primary concern, and Open AI was founded by a small group of eccentric billionaires motivated by a contrarian research hypothesis. Or maybe that was once the ostensible excuse, and now it’s just a regular startup that bootstrapped talent agglomeration through hype.

I’m not sure, and I very much hope to read the history of OpenAI once someone (or something) writes it.

What about the development in R&D?
Although the dedicated in-house lab is dead, corporate R&D spending is not.

From Nicholas Bloom in his Conversation with Tyler:

The share of R&D in the US and Europe… funded by the government has been declining over time. In fact, in the US, when you go back to the '60s, roughly two-thirds of it is funded by the government and one-third by private firms. Now it’s the reverse.

According to the NSF, it might be more like 75% private firms:

(I assume “private firms” just means non-governmental, as opposed to “firms not listed on public markets”.)

So yes, corporate R&D spending is very high, but remember that it’s a broad category.

When I think about Bell Labs, I think about Claude Shannon’s Information Theory, a leap in basic research that powered the information age, althought it wasn’t invented for any narrow purpose.

In contrast, the D in R&D stands for “development”, and won’t yield this kind of fundamental breakthrough. Take a look at Google’s 10-K. The costs are broken down as:

• Cost of revenues
• Research and development
• Sales and marketing
• General and administrative
• European Commission fines [for antitrust violations]

As described Quartz, “Much of those costs [of revenues] were from the fees Google pays companies like Apple to be the default search engine on iPhones and other devices, which are called traffic acquisition costs.”

So the R&D line refers to Google’s actual published research and the cost of cutting-edge projects like their Quantum Computing initiative, but it’s also just the cost of hiring engineers to work on ads. So yes, corporate R&D spending is high by some measure, but a lot of it is development, and a lot of that development has nothing to do with what we think of as research.

Exactly how much is research and how much is development? I’m not sure, and in some cases, it’s not even clear where you would draw the line.