The confidence to deliver an infrastructure portfolio

Bent Flyvbjerg developed a method to account for optimism bias called reference class forecasting (RCF). Simply put, RCF compares the estimated cost of a proposed project with the cost performance of similar projects, and adjusts the estimate accordingly. Suppose the estimated cost of a new intercity rail line is €2bn. A survey of ten similar projects in comparable jurisdictions has identified that the actual cost of delivery was an average of 39% higher than the original estimates. This suggests that the actual cost of the proposed rail line is likely to be closer to €2.78bn. This revised estimate should form the basis for the investment appraisal.

In fact, Flyvbjerg has identified that rail projects do indeed have a mean cost overrun of 39%. Clearly, it is then prudent to ensure that estimates are adjusted sufficiently to enable fair appraisal and ensure sufficient funds to complete the project. Governments are increasingly conscious of this. The 2019 Public Spending Code notes the impact of optimism bias. All else being equal, the consistent implementation of this practice since 2019 should lead to a significant increase in the estimated cost of projects when compared to their pre-2019 peers, while significantly reducing overruns. Better data leads to better decision-making.

Project cost estimates are often presented in terms of the probability of the actual cost falling within a certain range. If a project’s estimated (and adjusted) cost is €2.78bn with P50 confidence, this means that we can be confident that the cost will not exceed €2.78bn 50% of the time. As the relationship between the estimate and the confidence level is not linear, in order to reach 90% confidence the cost estimate would have to increase dramatically. In short, this means that it is prohibitively expensive to account for all risks and so a balance is required. If a portfolio of ten similar rail projects is proposed, it is unlikely that the worst case scenario would materialise for all ten projects. If the P50 estimate for each project is informed by an appropriate reference class, their actual costs should be distributed around that estimate. 

With a strong pipeline of rail projects, including MetroLink, Dart+, Cork Commuter Rail and the ambition set out in the All-Island Strategic Rail Review, there is an opportunity to transform public transport in Ireland. Critical factors for the success of that transformation include: reliable estimates, proactive risk management to keep costs down, and a focus on benefits realisation. Perhaps the most important success factor of all is learning from others. The Madrid Metro is a stand-out example of how to deliver excellent infrastructure and value for money. As we set out on our journey of transformation, we must look abroad to what has been done well and what could have been done better.

We are here to help you

Successful delivery of capital projects requires sound decision making. Experience tells us that decisions made in the early phases of a project have the greatest impact on outcomes. Decisions are often undermined by unknown unknowns. By using techniques such as reference class forecasting we can help you to make better, data-driven, decisions and achieve better project outcomes. If you would like to learn more, please do get in touch with us.