Decision making in the Ideation Toolkit

The Ideation Toolkit (from Keen Engineering Unleashed) is a collection of information that engineering students can use as they develop an idea for a product.

Although this toolkit includes the Analytic Hierarchy Process (AHP), I suggest that educators use multi-attribute utility theory (MAUT) instead of AHP.  I have been teaching engineering decision making for many years, and, although my textbook includes both AHP and MAUT, I have found that engineering students find MAUT easier to adopt and use correctly.  Using AHP in a rational way is more difficult than it looks, whereas MAUT is more straightforward for making decisions when the alternatives have multiple criteria that need to be considered (e.g., cost, strength, durability, etc.).

Scary Spider Chart

Spider chart.  Creator: redacted.

The spider chart shown in this post came from a dissertation that studied multi-agent systems. A spider chart is also known as a radar chart, and writers use it to graph multivariate data.  In a typical example, each alternative has a polygon that connects the points that represent its performance on multiple measures, with one point on one spoke for each measure.  It can be effective to show how one alternative performs well on one or more measures and another alternative performs well on another set of measures.  In that case, the polygons for these alternatives will have different shapes.
Conversely, the polygons for two alternatives will be very similar if the performance of those alternatives is similar across the different measures.

Unfortunately, the spider chart shown here has reversed the typical use.  There are two alternatives (adaptive risk and fixed risk) that were tested under three scenarios (low, medium, and high workload).  In this chart, there are six spokes, one for each combination; a typical chart would have six polygons.  Instead, there are three polygons, one for each measure: profit, completed percentage, and failed percentage.  The last two measures always add to 100%, so one of them is redundant.

Thus, there are only twelve useful data points (two measures for six combinations). The data-to-ink ratio is very low.  Given the small amount of data, a simple table may be the best way to convey this information.

The purpose of this spider chart is to show how the two alternatives compare on the performance measures, but this chart makes that comparison very difficult. A reader normally relies upon the slope of a curve (either positive or negative) to determine how performance is changing, but that will not work here because the different scenarios have different orientations and one performance measure is the complement of the other. 

Because there are effectively two performance measures, a two-dimensional scatter plot (with appropriate labels) would have been appropriate.  The second chart (which I created using the same data) is a possibility; this makes the change from fixed risk to adaptive risk more clear, but it still has a low data-to-ink ratio.

Two-dimensional scatter plot.  Creator: Jeffrey W. Herrmann.