Associations and data guided organizations of all types know that important business decisions should not be made based only on intuition, tradition, or politics. There is more data available than ever and an increasingly accessible tools to collect, analyze, and visualize it. But even skilled data analysts with the right tools and complete, clean data may reach incorrect conclusions. Why is this?
One reason is that, as humans, we’re all susceptible to Cognitive Bias – mistakes in reasoning or decision making caused by the tendency to have our decisions and conclusions influenced by attitudes, preferences, or beliefs. Because it’s a natural part of human thought and processing, we have to control bias to arrive at valid conclusions.
In an interesting session at Tableau Conference 2015, Jock Mackinlay (VP, Research and Design at Tableau), hosted session offering insight on how to combat cognitive biases on the road to data guided decisions. Cognitive biases are generally grouped into three categories:
- Social Biases – how we explain behaviors and qualities of ourselves and others. For example, self-serving bias causes us to take more responsibility for successes than failures. False consensus causes us to overestimate the degree to which others agree with us.
- Memory Errors cause us to remember things incorrectly. Hindsight Bias causes us to see past events as more predictable than they actually were – the “I knew it all along” effect. Consistency Bias causes us to view past beliefs and behavior as consistent with our current beliefs and behavior.
- Decision Making, belief, and behavioral biases – these are the types that are most relevant to Association Analytics – the biases that lead to incorrect conclusions from data. Examples include:
- Confirmation Bias – searching, emphasizing, or remembering information that confirms our original hypothesis or preconception, while ignoring or downplaying information which contradicts it.
- Illusory Correlation – perceiving a relationship between events that are unrelated
- Clustering Illusion – perceiving clusters and patterns that don’t exist, usually by overestimating the significance of a few data points.
- Base Rate Fallacy or base rate neglect – ignoring base rate information and focus on specific information (a certain case).
- Congruence bias – The tendency to test hypotheses exclusively through direct testing, instead of considering possible alternatives.
Often these biases result from our natural tendency to simplify information processing. There are so many decisions to make and problems to solve that we’ve developed mental shortcuts, called Heuristics, to cope with vast amounts of information. Heuristic techniques allow decisions to be made quickly and efficiently, almost automatically, while minimizing the amount of information processing in the brain. Some examples of heuristics we use in everyday life include:
- Common Sense – when a solutions seems clear or straightforward based on limited observation.
- Rule of Thumb – allows us to reach a conclusion based on a simple rule without exhaustive research.
- Authority Heuristic – causes us to instinctively trust in the judgment or expertise of someone based on their position (doctors, police, etc.)
- Affect Heuristic– allows quick decisions based on emotion, commonly referred to as “going with your gut”.
Heuristic techniques are a great benefit in some cases and a limitation that has to be combated in others. For many decisions a logical, analytical approach is impractical – we don’t have the required data, we value speed more than accuracy, or it just doesn’t matter that much – the consequences of being wrong are not severe enough to justify the effort. Other decisions, such as those relate to business strategy, products and services, etc. can have major consequences and demand a greater attention to the reliability of these techniques and the cognitive bias they lead to.
One of the interesting things about cognitive biases have been studied extensively and we know they show up in predictable and systematic ways. But even when an intelligent person is aware of a bias, it can still be very difficult to overcome. What is the best method of overcoming cognitive biases that we are all susceptible to?
Generally, it involves subjecting your decisions, conclusions, and underlying data to tests, trials, and independent review and validation. For Association Executives and Staff, this means two simple but very important things:
- Be Skeptical – compensating for cognitive bias means exploring your data thoroughly, even after an initial conclusion is reached or hypothesis is confirmed. When testing, don’t only test the specific conclusion directly, but consider alternatives.
- Share – Even if you’re confident in your conclusions, share them with peers for review and validation. They may see mistakes or bias that would be impossible for you. If possible, it’s generally a good idea to share the raw data in addition to the finished product.
