This piece was written by André Corrêa d’Almeida and Max Mauerman. André Corrêa d’Almeida is adjunct associate professor of international and public affairs at Columbia University, editor of Smarter New York City: How City Agencies Innovate and founder of ARCx‑Applied Research for Change. Max Mauerman is an economic researcher, writer and current student at Columbia’s MPA in Development Practice program.
City governments are catalysts for sustainable economic growth. This branch of government is most directly involved in citizens’ daily lives, given they manage investments in transit, infrastructure and education that form the backbone of all kinds of private activity. They are often able to act and adapt where larger-scale governments cannot, are more responsive to constituents and less hobbled by institutional constraints.
At the same time, municipal governments tend to be bound to standards of transparency and accountability in procurement that the private sector is not. In the US, such standards developed as a corrective measure against the graft and patronage of the political “machines” that controlled many city governments until the early 20th century. Good governance campaigners successfully pushed for laws like open bidding for government contracts in an effort to disentangle public management from private interests.
• Want to write for us? Take a look at Apolitical’s guide for contributors
However, the rules governing municipal procurement have not evolved as quickly as the accelerated nature of the economy. Today’s city governments are making investments not just in conventional infrastructure, but in boundary-pushing, complex technical projects, as documented in my recent Columbia University Press book “Smarter New York City: How City Agencies Innovate”.
Existing procurement laws and procedures weren’t built to handle such projects, which require iteration, experimentation, adaptation, close collaboration between the public and private sectors, and co-development of the design of the final product.
What is the price of this incompatibility, and how could it be avoided? To answer these questions, we have developed a framework to categorise and measure the costs of lost collaboration between city governments and their private sector partners. Broadly, these costs fall into four categories:
- Restraints on innovation
- Inefficiencies in the bidding process;
- Ex-post adjustment costs, and;
- Socio-political opportunity costs.
1. Restraints on innovation
From the very start, would-be public projects must navigate a procurement process that was not designed to accommodate the contingencies of technological innovation.
Most city governments require requests for proposals (RFPs) using an open bidding process where competing contractors are evaluated on the basis of the price they can offer for a pre-specified piece of work. This process was designed for projects with a well-defined output, a relatively homogenous product and a plethora of potential suppliers, such as road construction.
• For more like this, see our cities newsfeed.
However, these assumptions break down in several costly ways when applied to urban tech projects. The first is the specification of a product or service without exploring a broader range of solutions. Too often the city defines the product or service based on past experience, rather than using an RFI (request for ideas) or RFPS (request for problem solutions) to engage its suppliers in an innovative exploration of options.
Too often the city defines the product or service based on past experience
Similarly, cities tend to act in isolation rather than pooling and sharing innovations, leading to inefficiencies and duplication of efforts. In simple terms, the antiquated RPF process for cities is static and fails to capture the eco-systematic nature of creating solutions that can evolve with technological innovations and the dynamic nature of urban life.
2. Inefficient bidding processes
Second, the pool of candidates for tech RFPs is often limited. By nature, urban tech projects tend to be highly specialised, and the scarcity of appropriate suppliers means that contracting firms can exert some degree of monopoly power. The usual efficiency justifications for a competitive bidding process don’t hold in cases like this.
This barrier to entry can keep small contractors out of the market entirely
This situation is made worse by the costs of navigating a byzantine government bidding process. The complexity of RFP portals has spawned a cottage industry of consultants that help would-be contractors get their proposal through layers of bureaucracy. Unsurprisingly, this creates a barrier to entry that can keep small contractors — more often than not the most innovative and flexible of all — out of the market entirely. This is a tremendous contradiction with the principles of home-grown innovation and economic development.
We have measured the costs of having a limited pool of suppliers in a couple ways. Quantitatively, we apply Joseph Stiglitz’s concept of monopoly rents in imperfect auction markets to estimate the markup that firms can impose from their scarcity. We have supplemented this with open-ended interviews with tech firms to understand the barriers they face in navigating the RFP system.
3. Adjustment costs
The other way that urban tech projects confound the procurement system is the indeterminate nature of their costs. Unlike conventional infrastructure projects, where the costs of materials, labour and contingency premiums can be set out with some confidence in advance, the implementation of urban tech projects tends to be an iterative process, with deliverables and timelines subject to frequent change.
The structure of compensation in supplier contracts isn’t designed to accommodate these sort of projects. Public sector procurement contracts tend to be “fixed-price” rather than “cost-plus”; that is, contractors assume financial responsibility for any expenses beyond the agreed amount, and have no flexibility to adjust for unforeseen costs. Similarly, if costs decline during the life of the project (e.g. due to technological efficiency gains), these savings often aren’t passed on to the buyer.
Even where cost-plus contracts exist, the bidding system does not necessarily reward the best suppliers. A system designed around minimising visible costs (i.e. profit margins) can actually create perverse incentives, as the suppliers with the lowest bidded margins are often the least reputable and cost-efficient ones.
Lack of communication between government buyers and suppliers also imposes adjustment costs. An incompletely specified scope of work can lead to cost overruns and delays if the client realises a need for new features partway through the implementation process — a common occurrence in tech projects.
We can measure these various forms of cost overrun by going to the books of contractors and analysing how costs evolve over the course of a government project. In particular, we are looking at “liquidated damages”, the financial penalty that municipalities can impose on contractors for undue expenses or delays, as well as the cost of renegotiating contracts.
4. Opportunity costs
Last, there are significant but often unseen opportunity costs to society from the inefficiencies described here. Delays and snags in rolling out vital systems, such as social service registration portals, or public security solutions to identify the sound of a gunshot in real time, impose a great burden on citizens.
Likewise, the costs of correcting for the flaws of the procurement process diverts scarce municipal budget money from other critical areas. Botched implementation of large public projects also erodes citizens’ trust in the city and erodes the city’s political capital.
The costs of legal rigidity are greatest for specialised projects with hard-to-predict costs
This framework, which will be detailed in an upcoming “part 2” to this post, can help us understand where cities are losing out in their investments and how to improve it. A key lesson is that the costs of legal rigidity are greatest for specialised projects with hard-to-predict costs — exactly the sort of innovative projects, in other words, that many cities are looking to invest in.
With this in mind, there is a need for adaptive contracts (e.g. cost-plus or cost-plus with a cap, co-development) between public and private entities that are better suited to path-breaking projects. Smarter contracts can help align the interests of the public and private sector providers.
Cities should also rethink the conventional bidding process for these sorts of projects, and instead look for systems that foster closer communication and collaboration between government and contractors.
For example, “Smarter New York City: How City Agencies Innovate” tells the story of how the New York City Police Department partnered with ShotSpotter to develop over time a flexible system capable of identifying the sound of a gunshot in real time.
Cities should look into dividing large urban tech projects into discrete pieces that can be handled by smaller, more nimble firms. Cities should also consider the ways that technology can built into conventional projects as an ancillary benefit, e.g. adding pollution sensors as part of road construction.
Cities should also continue to embrace capital building practices where they partner with less established and more innovative companies who will offer more forward-looking and evolutionary solutions.
These systems require a greater degree of trust in suppliers, which comes with risk, but their potential rewards are great. The reforms are needed to accelerate innovation in urban settings and ensure that cities and local governments remain engines of innovation and sustainable growth. — André Corrêa d’Almeida & Max Mauerman
(Picture credit: Unsplash)