Dunning-Kruger is misunderstood (or the importance of going to the source)

Typically, the Dunning-Kruger effect is understood as the tendency of people to highly rank themselves in areas in which they have no competence.

What you’d glean from the wording is that unskilled people rank themselves higher than skilled people.

Well, the data and graphics in the study (which you can get here) don’t lead to that conclusion. Indeed, it doesn’t address how you could discern someone who is highly skilled or not based on their self-asessment, which is what the conclusions in the second paragraph, the conclusions we’d take from the wording used to explain the effect, imply.

What does it state, though? That unskilled people are badly calibrated, and when they get better at a particular skill they can better predict their own performance.

In fact, there’s a sweet spot in the third quartile of performance where people seem to be best calibrated, whereas the best in a particular field tend to underestimate their ability.

There is another aspect covered in the paper which may be considered: that unskilled people tend to underestimate other people’s performance. So, if we don’t know the skill level of someone in a particular area, we shouldn’t really trust their assessment of other people. This is very different a conclusion from what we see in our day-to-day interpretations of this paper; because people who talk about papers and think they understand them don’t always do – they may be miscalibrated and thinking their understanding is better than it actually is.

Thus, the study steers us towards a peculiar worldview: in order to trust someone’s opinion on a subject, we should have their skills in high esteem. But in order to have a good calibration of their skill level, we should have a certain skill in the specific area. This is an argument for scholarship and erudition, learning all we can, even if it is just enough to decide who we trust. The best way to build that understanding in matters of scientific breakthrough, critical thought, situational analysis and data analysis is to go to the source.

To help everyone augment their level of competence in this area and better decide whether they choose to trust someone in a particular area, such as interpreting well-known phenomena or re-casting what we think of a particular event (or paper), it is a good practice to leave the sources used. So go, read the paper (14 pages only), and start a healthy discussion. It is the only way to improve our skill in the area, and learning who we can trust to gain new knowledge for us.

Leave a Reply

Your email address will not be published. Required fields are marked *