Why Data-Driven Roadmaps are Vital
Overruns remain a big problem in the software development industry. A study by McKinsey & Company found that businesses have IT projects running at 45% over budget on average while delivering 56% less value than the original forecast. This inefficiency leads to millions of dollars in losses and threatens the sole company existence for 17% of businesses. [1] The study indicates lack of business focus, unrealistic schedules, and unaligned teams as the major causes of project failures.
In software companies, roadmaps play two important roles. They show the list of business initiatives the company is focusing on and present how many of those initiatives they aim to deliver within the next few months. Such representation serves multiple purposes:
- Alignment of teams and leadership on the company’s priorities, backed up by financial and timeline constraints
- Helicopter view and scoping on strategic initiatives
- Prediction on future needs for resources and scarce talent
While such representation is critical for a company’s performance, very few companies dedicate the necessary amount of time on creating roadmaps. Usually no formal process is introduced, and roadmap creation is performed by surface data and intuition. [2] Items are sketched on a whiteboard or in Excel where:
- Engineering teams may not be present or participate in validating the commitments
- Items are positioned manually without proper adjustment for risk buffers, strict control of work-in-progress, or dependencies
- Scope and assumptions are not recorded and communicated verbally
While intuition-based approach looks simple and straightforward on surface, studies show that it is highly vulnerable to a cognitive bias. People systematically underestimate the time, costs, and risks of future tasks while overestimating their benefits. [3] A better approach would be using more reliable planning methods and developing proper governance over the planning process [4] while maintaining short delivery cycles to avoid waste. [1] Such a strategy can improve deliverability and performance. [1]
Building Roadmaps with a Lightweight, Data-Driven Approach
The solution that will be described further was tested in a fast-paced and highly-volatile environment. It was used to deliver numerous successful projects that involved international cross-team collaborations and tight coordination with product and project managers. Every project contained different levels of uncertainty, as well as multiple internal and external risk factors. All were delivered on time, including when delays were negotiated due to priority shifts, and without major disruptions. Some projects were mid-size (3-4 months), some required a longer timeline (1-2 years). This solution helped the company avoid risky investment decisions, properly allocate the load between development teams, and build trust with stakeholders. Not once was the process rigid or overly tedious, it only required basic effort to keep the governance.
First Step: Define the Portfolio
To replace an intuitive process with a more structured one, we need to take a step back and define a portfolio first. The portfolio defines the signals worth exploring. Similar to scientific or VC planning, the hypothesis, or signal, is defined as being worth investment and testing. Once enough information about a particular signal is collected, it can be either reinforced or cancelled to leave opportunity for new developments.
In the era of AI, this becomes particularly relevant. Some signals can be tested in a single day, while others may need more steps and a month of research and development. Which signals are worth exploring? Without portfolio modelling, this question isn’t possible to answer properly.
Now is when roadmaps come into play. The roadmap should serve not just an execution plan, but first of all, as a proof of portfolio feasibility.
Build the Portfolio
The portfolio should contain current business hypotheses the company wants to test with clients. Normally, we talk about initiatives and features that represent underlying business ideas. Each feature or subset of features could be prioritized against their potential ROI using different product management frameworks (such as RICE, Kano, etc).
Collect the Data
All requirements, dependencies and other inputs should be carefully collected, verified with teams, and recorded for current and future reference. Some technical inputs, such as high-level estimates, must include a level of uncertainty and assumptions. The data shouldn’t necessarily be 100% or even 50% complete, but if there is any level of risk, it must be recorded. Different techniques could be used for estimation, such as using historical data, planning poker, etc. The book Software Estimation: Demystifying the Black Art by Steve McConnell provides excellent insight into highly effective methods of estimation, even for incomplete or lost reference data [5].
Do the Sequencing
Once the data is available, the next important step is execution sequencing. All data pieces must be evaluated:
- Which scope do we assume for each feature?
- Which teams will be performing the features?
- Are there any dependencies?
- What is the risk buffer? How does the buffer correlate with uncertainty and risk appetite?
- How calendar days allocated for the tasks relate to the teams’ availability and velocity?
These steps will ensure teams’ productivity by limiting work-in-progress and minimizing delays caused by local disruptions [6].
The sequencing is a tedious process so it’s better to use specialized project management tools to keep the process bias-free and lightweight. One option is to use Excel, MS Project, and Wiki to calculate dates and store assumptions. Alternatively use probabilistic planning tool Deep Planner for maximum consistency and automation.
Perform Alignment
Developers, stakeholders, managers and other parties must be invited into the roadmap alignment. There could be multiple rounds of reviews where parties will be able to verbalize gaps in data, scope preferences, or claim ownership of features. This is also a good time to reassess the portfolio hypothesis in case it wasn’t proven practical. A properly aligned roadmap serves both as a proof of portfolio visibility and an execution strategy.
Execute and Track the Status
The roadmap can be executed using normal short development cycles, like sprints. Measuring the portfolio progress should be connected to measuring the progress and completion status of the underlying initiatives, features, and their dependencies. This should be an ongoing and frequent process-–ideally, synced to the short development cycles—to address the potential risks and make any necessary adjustments early. To update the roadmap, follow the same steps, starting from updating the portfolio inputs, and proceed downstream.
Summary
Roadmaps treated as static promises or intuition-driven sketches pose major risks for companies’ performance and deliverability. Organizations can avoid the trouble by validating portfolios using a lightweight approach featuring structured data, explicit assumptions, and good alignment. The goal is not to eliminate uncertainty but to make it visible and manageable, thereby significantly reducing the risk.
Links
[1] – Research “Delivering large-scale IT projects on time, on budget, and on value” by Oxford.
[2] – Research “Decision-making in Software Product Management: Identifying Research Directions from Practice” by Andrey Saltan, Slinger Jansen and Kari Smolander
[3] – Research “Exploring the “Planning Fallacy”: Why People Underestimate Their Task Completion Times” by Roger Buehler, Dale Griffin, and Michael Ross
[4] – Research “Eliminating Bias in Early Project Development through Reference Class Forecasting and Good Governance” by Oxford
[5] – Steve, McConnell. Software Estimation: Demystifying the Black Art (Developer Best Practices) . Pearson Education. Kindle Edition.
[6] – Research “Implementing critical chain to improve product development performance” by Stanford.
