Dancing with the Stars
The rules for scoring ballroom dance performances are universal. The criteria for deciding which projects to invest in? Not so much.
Have you watched the television show Dancing with the Stars? The judges compare dance performances by scoring each dancer against standards such as posture, timing, and body line. In a similar way, organizations can vet project ideas using standard criteria.
Organizations need to evaluate the value of projects for many reasons. For example, most have more potential work than resources and time. They need a way to give a green light to the best projects and decline the rest. They may also need to construct a portfolio of projects, decide how to handle a sudden diversion of organizational resources, or figure out how to fit in an urgent new initiative among existing projects.
Portfolio management provides a useful framework for this, allowing decision-makers to compare the value of proposed and active projects using standard evaluation criteria that apply to all projects.
How to Score Projects
Evaluation criteria for projects must embody what really drives an organization’s business or mission. Design them so projects that will deliver the most value receive the highest scores. Each organization’s drivers are unique, so evaluation criteria must be customized for the organization, although there are similarities across industries.
Some important criteria can be quantified using the language of financial benefit, for example payback period, expected revenue (stated as NPV), or cost reduction. Other important things are harder to quantify. Instead of trying to reduce them to dollars, score them like dance judges holding up score cards.
Here are some examples of non-financial criteria from real organizations that I know.
Aim for a Goldilocks number of criteria. Using only one or two criteria will miss important considerations. For example, you’ll see a lopsided picture if you compare projects by looking only at expected revenue. What good is a project that might produce maximum revenue but has sky high execution risk? At the other extreme, evaluating too many criteria becomes unwieldy, leading to analysis paralysis and feeding the data monster. Three to eight criteria are often just right.
Simple Scores and Good Judges
Scoring should be simple. For example, use a scale of 1 to 5, clearly defining what each point represents so raters score the projects consistently. Designing simple but useful scales can be difficult. Points on the scale must lead to meaningful distinctions among the options being evaluated while avoiding common rater biases such as the central tendency. Clear definitions help ensure objective, aligned decision-making, especially with multiple raters.
Here’s a real-world example. Improving workplace culture was a priority for a certain company, so it defined a scale to measure a project’s impact on culture. A score of 1 meant the project was likely to have a negative effect on culture, such as increasing employee turnover. A score of 5 meant the project was likely to increase retention by a certain amount. These scores helped determine which projects to focus on.
Remember that criteria – whether financial or scored – can’t capture everything important. Selecting which projects to invest in is not just a numbers calculation. The best use of criteria is to provide guardrails for a constructive discussion among the decision-makers. Leave room for discussion and expert judgement.
Ch-ch-changes
Counterintuitively, less can be better. Shared decision-making about projects is a multi-year journey for most organizations. Early in that journey, a detailed and prescriptive scoring framework can provide tight guardrails that help decision-makers stay focused and consistent. As the organization matures, loosening the guardrails encourages more creativity and constructive discussions about the options.
A good portfolio management system helps you separate the best projects from the rest, striking a balance between structured analysis and thoughtful discussion. Take a fresh look at your project selection process and refine your criteria so your portfolio of projects supports both short-term goals and long-term success.
I teach and consult about highly effective projects, portfolios, and teams. You can contact me at [email protected].