When Free and Open Source Software isn’t Free
What motivates businesses to pursue open source, and is the U.S. government getting a bad deal?
As any student learns in an introductory economics class, there is no such thing as a free lunch. And yet, many businesses contribute extensively to open source libraries, choose to open source entire product lines, and even pay other companies to improve their open source offerings. The U.S. government, for its part, often encourages vendors to use only free and open source software (FOSS) components in their software offerings (versus paid commercial software). So where is the hidden cost?
To understand why FOSS isn’t free, we first need to understand the incentive structure motivating businesses to pursue open source. A helpful corollary is the concept of complementary goods, such as gas and cars. When the price of gas goes down, the demand for cars goes up, which subsequently increases the price of cars. This means businesses are motivated to find a way to make complementary products cheaper.
Put another way, businesses are incentivized to commoditize their complement.
Joel Sapolsky makes this point is his classic 2002 post on open source: “I noticed something interesting about open source software, which is this: most of the companies spending big money to develop open source software are doing it because it’s a good business strategy for them, not because they suddenly stopped believing in capitalism.”
He goes on to give the example of IBM spending millions to develop open source software not out of altruistic motives but because IBM was becoming an IT consulting company. “IT consulting is a complement of enterprise software. Thus IBM needs to commoditize enterprise software, and the best way to do this is by supporting open source.”
Palantir has more than 200 open source projects. The investment of engineering hours into these projects makes them expensive. Palantir pursues this strategy because we’re either commoditizing a complement that enables more budget to be available for our category of software, or we’re reducing friction to a sale by delivering anti-lockin mechanisms. For example, in 2012 Palantir paid for PostGIS to build open source geospatial features because we wanted the budget customers would pay for databases backing Palantir's software to instead be available for our software.
In today’s Department of Defense context, open source is maximally aligned with companies that bill hours and put butts-in-seats rather than sell software. Open source-only solutions enable traditional, labor-based Systems Integrators to commoditize their complement — commercial software — and retain maximum share of government budget for billable hours.
To be clear, there is nothing nefarious about this strategy. In a balanced market, this is a powerful way to drive competition and price-performance of commercial software companies. It rightfully puts the onus on Palantir and other companies charging for software products to prove outsized value over freely available alternatives. This spurs innovation.
The problem arises when the Government kills the market mechanism by prohibiting the use of commercial software in its solutions. When the Government hands Systems Integrators the budget for billable hours rather than making them compete with commercial alternatives, it rejects the American belief in the value of competition and markets. USGIF’s recent NRO Industry Advising Working Group report, “Reducing Barriers to Uptake of Commercial Technology,” to which Palantir contributed, identifies several recent government solicitations discouraging the use of commercial software:
“We are seeking innovative approaches that include…reduced dependencies on licensed software products through the use of free and open-source (FOSS) software.” “Attributes for [the desired] architecture” include “maximizing use of FOSS.”
“The Contractor shall design and integrate to preclude long-term dependence on closed/ vendor unique or proprietary interface standards, technologies, products or architectures.”
“The contractor must obtain NRO permission before delivering software under this contract which incorporates any other software that is not free and open source.”
The relentless focus for any product or system involving software should be the end-to-end delivery of a solution to drive an outcome. Outcomes should take precedence over requirements that serve as poor proxies for what we hope will translate to outcomes. The success of cloud computing as-a-service over open source alternatives (e.g, OpenStack) demonstrates how incentives affect the pace of innovation. Open source innovates at the pace of the party commoditizing something (at the limit). Commercial software as-a-service innovates at the pace of winning the marginal customer; it forces the software developer to feel the pain of cloud optimization, performance, DevOps, and feature innovation.
Users want a service, not a file with code in it. Open source competition for Windows did not work. No one was going to win the consumer or enterprise market with Linux. There were niches of developers using different versions of Unix, but there was no obsession with creating a product. Ultimately, Apple did win the market by building an Operating System on top of OpenSource BDS to make MacOS. It commoditized its complement to enable custom services and products (e.g., App Store, Apple TV, iCloud, and the hardware required to leverage those services).
For a humorous example, here is an infamous Hacker News commenter who argued in 2007 that Dropbox as a product didn’t make sense because anyone could “build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem.” This of course ignores the millions of users who wanted the convenience and white-glove experience of having a fully managed file storage system whose security and updates they didn’t need to worry about.
In its quest to commoditize its complement, Systems Integrators have succeeded in creating a narrative that only FOSS liberates the Government from vendor-lock. But the only way to build a mission system or product from FOSS components is by implementing custom code. Per the USGIF report, “ironically, the complexity and bespoke nature of FOSS-based systems drives outcomes the Government is trying to avoid: increased technical debt and vendor lock to the developer.” For example, one government agency aggressively avoided a popular commercial SIEM solution only to find it was locked into the custom alternative it built.
The Government’s preference for exclusively FOSS solutions does not save money. It just shifts the expense from commercial software to labor. Zooming out, this hurts American prosperity, since $1 spent by the Government on labor is worth ~$1-2 of wealth in American retirement accounts. That same $1 is worth $5-20 when spent on software because the market values the revenue of software companies at 5-20x the revenue of the traditional primes. The increased market value generates more taxes through capital gains, which funds the very Defense enterprise.
American prosperity - growing the economy - underpins our nation’s security.