A previous article examined how to use Promax to recreate the model for decisions involving multiple objectives as described in the book “Decision Analysis for Management Judgment” by Paul Goodwin and George Wright (1991).
In the example we used a real-world scale for some of the criteria (for example, office floor area in ft2) and a preference scale for criteria where real-world scales were difficult to define. The problem with preference scales is that they are unique to the group involved. The chances of a completely different group choosing the same preferences are remote. This means that the result cannot be adequately explained or justified and it also cannot be recreated. This may be perfectly acceptable for a decision where there is a single decision maker but it is not acceptable where there are multiple decision makers – for instance in large organisations where decisions are approved and reviewed at multiple stages.
This article looks at another type of scale that can be used for those “difficult-to-measure” criteria – namely textual scales.
Textual scales are ones where the degree of preference is explained in words rather than numbers. They are created in a similar way to preference scales except that the reasoning behind the position on the scale is explain in objective terms. See an earlier blog for the differences between objective and subjective information.
For the criteria being discussed, write, as clearly as possible, the best imaginable score. The addition of real examples that people would recognize is well worth the effort. Then consider the worst imaginable. Then imagine what lies between the best and the worst. Don’t think in terms of position (e.g. halfway between best and worst) but more in terms of discrete differences. They should be something that you can describe as being a step-change difference. Then, create as many intervals as you need that will adequately segment the scale. For example, office image could be explained thus:
T
|
This office compares with the best in the
world. As a benchmark the TBWA Hakuhodo offices in Tokyo.
|
H
|
This office is just about the best in the
area. As a benchmark the “Hatton Gardens” office block.
|
K
|
The offices in “Kent Boulevard” are the
benchmark here
|
L
|
The offices along “Link Road are the benchmark
here
|
M
|
The offices along “Mellor Drive” are the
benchmark here
|
I
|
This office is just about the worst in the
area. The offices in “India Road” and “Jason Place” are the benchmark here
|
P
|
This office compares with the worst imaginable. A
dark, dingy, very run-down office which is particularly uninvited. For
example you’ll smile smugly to yourself as you imagine people whose offices
are much messier than yours. You’ll stretch your arms out and luxuriate in
the fact that you can’t even touch both walls of your 8×8 cubicle
simultaneously as you view picture of a worker toiling in a space smaller
than a closet in a one bedroom apartment.
|
Figure 1: The Promax scorecard display once descriptions have been defined
There are some top tips in creating textual scales:
- Do not use numerical scales (e.g. 0-10) even if you explain what a 10, or a 7 or a zero is. These scales are prone to people “gaming the system”. This means that since people know that a 7 is a better score than a 5 they will persuade others that the rationale behind giving the scores makes sense. It may make sense at the time, but later on you will struggle to reconcile the scoring with the facts. So, people can quite easily generate the result they always wanted even though apparently the methodology was fair and transparent. That is why techniques using numbers such as Kepner-Tregoe, the Pugh Matrix, AHP etc. should be avoided – they appear robust and scientific but are anything of the sort.
- Do not use Very Good, Good, Average, Poor, Very Poor even if you explain what “Good” means. Again, it is too easy to game the system because you know that a Very Good is better than Good.
- Do Randomise the scale. In the same way that a well-conceived survey randomizes it’s questions in order to reduce bias so you should do this for textual scales. This means not ordering the scale from good to bad. Order them in a different way otherwise, whatever the description, you end up in the same trap as for the previous two tips – people can game the system because they know what scores will generate the result they want.
Once you have created your scale you need to value the points on it. The best imaginable is worth 100 and the worst is worth 0. The others will lie somewhere between 0 and 100. Remember, as in the mapping done for real-world scales the mapping does not need to be a linear, straight line. You may decide that anything with a better image than “Link Road” does not add any more value so you end up with more of an s-curve.
Always keep the scaling separate from the mapping. In Promax, when using textual scales you need to map the scale against the value as part of the process.
