How Many Criteria?

I'm often asked "What's the best number of criteria to use?"

As a matter of simple arithmetic, the more you have, the less impact each one will have on the decision. So if you have 4 criteria and each one is weighted equally, then each will contribute 25% to the decision. If you go up to 10 criteria then the contribution of each drops to 10% and so on. When you have over 10, even doubling the importance of a single criteria will make little difference to the final result.

However, a better way of asking that question is to turn it around and not query how many criteria should I have but what criteria should I use. The number that results is the number you need.

In determine what the criteria should be, it is never a good idea to simply brainstorm within your team to come up with ones you think are differentiators. A far better approach is to use a hierarchy of objectives. Here you consider the overall objective or mission you're trying to accomplish. Then question what lower level objective supports the main mission. Finally what sub-objectives or measures support those. It's these lower level objectives - ones that are measurable - that should be your criteria.

For most evaluations you will easily end up with more than 4 or 5! However, that isn't a problem as long as you weight them at the right level. In the diagram above, there are 4 main objectives contributing to the Mission so we weight at that level instead.

In Promax this is called "Top Down" weighting. Go to the weights tab and click the Top Down button. An arrow will appear next to the relevant weight set. This means the calculations will be based on the weights of the topics or nodes (the red boxes) rather than criteria (grey boxes).

By default, in top down weighting, the criteria below each topics are equally weighted. So if you have more criteria in one topic than in another this may lead to unforeseen results. You can easily alter them to reflect a more correct weighting but remember that it's the weights at the topic level that have the final say.

Conclusion

Try to develop criteria based on objectives rather than randomly brainstorming them. That way you can demonstrate how each option contributes to the overall mission. You can much more easily show in what areas it excels and where it is weak.