By Phil Cowperthwaite
Illustration: Gary Clement
There are many inherent barriers to making good judgments, but they can be overcome with some effort and proper safeguards
Professional skepticism is a cornerstone of audit quality. It defines the quality of each audit judgment and, to a significant extent, is the essence of the audit. The connection between skepticism and professional judgment is complex, which research across many professional disciplines is proving. The relationship needs to be better understood to improve our audits.
Skepticism is a questioning attitude; exercising professional judgment is a process required in forming an opinion of value. Having a skeptical attitude is essential for auditors when analyzing evidence, looking for inconsistencies in data and considering explanations from management. That attitude shapes the actions required in forming a professional judgment. And auditors need to make every judgment count on every audit they undertake.
There is a significant body of research that shows our brains are prone to make cognitive judgment errors in a variety of circumstances. Our lightning-fast intuition can lead us to dangerously incorrect assumptions of which we can be completely unaware. Research also claims that serious cognitive errors are a part of our human DNA, hard-wired into our physical being. Our brain, it turns out, naturally works against skepticism, and for some very good evolutionary reasons. As we can’t be reprogrammed, the best we can hope for is awareness. This is an issue that affects more than just the auditing profession. As such, it is instructive to look at other disciplines to obtain insight into how these barriers affect making good judgments.
Unconscious cognitive errors: barriers to making good judgments
Over the years, cognitive psychologists have identified many inherent barriers to making good judgments. These barriers are especially relevant to professionals — that includes auditors — making difficult judgments under stressful situations. Our intuitive thinking processes can unconsciously dull or even eliminate our professional attitude of skepticism in certain circumstances. If an issue does not activate an auditor’s skepticism button, then it won’t be the focus of professional judgment.
While these barriers can be overcome, to do so requires being conscious of our natural traits to act in certain ways and the need to apply effort to get past them.
To help compensate for these natural traits, it is helpful to know of at least three barriers to making good judgments that are likely to be very relevant to professional auditors, regardless of their position in a firm or area of specialization.
Research shows that people are more confident in their ability to make good judgments than they should be. We make snap judgments based on past experience, which is not helpful during an audit when this experience is not relevant to the decision at hand.
For example, many auditors think they have a good feel for selecting sample sizes and interpreting the results of their tests (Thinking Fast and Slow, Daniel Kahneman). Many highly educated professionals in other fields with training in statistics think the same way. Research demonstrates that most people are lousy intuitive statisticians; we just don’t naturally process information this way.
Instead, we are prone to jumping to conclusions regardless whether the size of the sample is statistically valid or not. This natural bias can be disastrous for an auditor relying on a statistically unsound sample. The natural reaction for an auditor is to be overconfident in accepting results based on prior experience, and this may be a dead wrong conclusion as significant variations from the norm are not uncommon in a much-too-small sample drawn at random from a population.
Another example of the overconfidence bias: upon seeing a set of facts, immediately reaching a conclusion based on prior experience — especially if it was recent. We naturally discount statistically remote outliers, gravitating to what we know was correct in the past. In fact, the remote outlier might be evidence of a serious and new problem. This failing is compounded by tight deadlines, tired professionals and a need to move on to the next crisis. Sound familiar?
A natural tendency to overconfidence can lead to seriously flawed judgments in any audit. But deciding to test larger samples and more fully explore anomalies comes with a cost — more time. This increase can be especially burdensome in a very small audit.
Starting points affect your judgments
Research shows that our judgment regarding the reasonability of an estimate is heavily influenced by the initial estimate provided (often referred to as an anchor by cognitive psychologists).
In one experiment, two groups of real estate agents asked to quote a reasonable buying price were shown different asking prices; one higher than the actual price and one lower. When asked about influences, both groups said the asking price was not a factor in their estimation. The agents shown the higher price recommended a buying price significantly higher than the agents who were shown a lower amount.
In another experiment, judges were asked to recommend a sentence for a petty crime after rolling a die that was rigged to show either a three or a nine, an event they knew in advance had nothing to do with their task. On average, judges who rolled a nine suggested a sentence of eight months; those rolling a three averaged a sentence of five months (Thinking Fast and Slow).
Replace an initial management inventory obsolescence estimate (the anchor) for the asking price of a house and you begin to see the possible impact on auditors evaluating management judgments. Research conducted with experienced auditors shows they have a natural tendency to start from management’s anchor instead of stepping back and judging an appropriate range from the underlying facts. This can lead to serious judgment errors (“Analytical Procedure Results as Substantive Evidence,” W.R. Kinney, Jr. and C.M. Haynes).
Our brains are highly susceptible to suggestion. While this statement might seem obvious, it comes as a shock that experienced professionals are as susceptible to suggestion as everyone else. Overcoming the power of suggestion takes concentrated effort. It requires a process to force the professional to step back and question something that intuitively seems acceptable.
Judgment bias resulting from availability of information
Our most recent memories also disproportionately affect our initial judgments. This can work two ways: current events or experiences can blow a small risk out of proportion and cause us to overreact, while a likely risk we have no recent memory of can be inappropriately discounted. Either way, our judgment will be biased based on recent memories. This can lead to judgment errors in audits in several ways.
For example, numerous studies show that recent events reported in the media can have a disproportionately large impact on our judgments (Thinking Fast and Slow). Earthquakes and other natural disasters frequently result in immediate increases in insurance prices, even though there is no evidence that the likelihood of a future loss has increased. Man-made environmental disasters also frequently result in significant and costly public policy actions that, in many cases, may be disproportionate to the risks involved.
On the audit front, risks associated with a new financial instrument may go undetected by an auditor and management team because of a lack of experience with the new product. This was a contributing factor to the surprise of Canada’s sudden asset-backed commercial paper market collapse in 2007. However, recent events such as problems in some sovereign debt markets may make auditors overreact to conditions in unrelated markets. Our natural judgment process is highly affected by readily available memories. It takes conscious effort to overcome this natural tendency.
Overcoming inherent cognitive bias
Unconscious cognitive errors are very real human traits. Usually we are not conscious of their effect on our day-to-day judgments. While these traits can result in efficient and reasonably accurate choices for most of our routine decisions, our unconscious bias is the enemy of skepticism in complex situations. Basing conclusions on insufficient evidence, an inappropriate starting point, and recent memories can get professionals into serious trouble.
Judgment bias can be overcome, but not without effort. Implementation of safeguards in every audit will help guard against inherent bias. Many of the larger accounting firms have developed strategies to heighten professional skepticism and help overcome judgment bias. These strategies are expensive to develop and to maintain. However, it is just as critical for smaller firms — and especially sole practitioners — performing engagements to adopt processes to overcome inherent judgment bias and heighten skepticism. These processes must be appropriate for small audit teams in order to be effective.
Up next month: processes for making good judgments in micro-entity audits.
Phil Cowperthwaite, FCA, is a partner of Cowperthwaite Mehta and a member of the IFAC’s Small and Medium Practices Committee
Technical editor: Ron Salole, vice-president, standards, CICA