Visibility bias
Judging only from visible opinions may lead away from the actual overall picture. People who do not speak up also hold information; what is visible is not necessarily the whole.
How it started
This concept grew out of the unease of hearing claims about taking diverse opinions into account while, in practice, observing only those who are loud and assertive and ignoring those who remain silent.
Mechanism
Those who speak up are a biased sample
People who speak loudly, assert themselves strongly or can bear the cost of speaking up are more likely to be observed.
Silent groups are not visible
People who have opinions but do not express them can appear absent from the observed data.
Visible opinions become representative
It is easy to mistake observed opinions for the opinions of the entire population.
Noise is amplified
Opinions expressed more frequently or forcefully appear more important and can have an outsized influence on judgment.
Typical cases
- Treating opinions on social media as public opinion
- Making meeting decisions based only on those who speak up
- Not considering people who did not answer a survey
- Treating a large number of complaints as dissatisfaction across the whole population
- Judging only opinions with many reactions as supported
Countermeasures
Account for unobserved groups
Start your decision model with the assumption that there are groups you cannot see.
Do not assume what silence means
Do not automatically classify a lack of response as agreement, opposition or indifference.
Correct for sampling bias
Check who can easily speak up and who is missing from observation.
Make the source explicit
Distinguish which group the opinions were observed from, rather than treating them as the opinions of the whole.
How to read a practical example
When only risk information is emphasized and used to justify a ban, it can lead to the simplification that danger means exclusion.
Example: something viable under certain conditions is banned across the board
Risks, safety, acceptable limits and the conditions under which something works need to be evaluated separately. But judging only from visible risk information can drop unobserved conditions and exceptions, making a blanket ban more likely.
One-line summary
What has been observed is important information, but it does not mean that what was not observed does not exist. Judgment needs to consider both what is visible and what is not.