In this episode, Dr. Azdren Coma explores how the Dunning-Kruger effect distorts self-reported skill ratings in hiring and surveys. A seemingly straightforward question like “On a scale of 0 to 10, how skilled are you in Microsoft Excel?” can produce misleading data because undefined scale points allow respondents to interpret them differently. Beginners often overestimate their abilities due to limited awareness of complexity, while more skilled individuals rate themselves more conservatively as they recognize what they don’t know—potentially reversing the ranking of candidates.

This episode breaks down the psychological mechanism behind the bias, illustrates it with real-world examples, and offers practical fixes for better data quality, including fully labeling scales, using clearer anchors, and shifting to task-specific questions. Ideal for HR professionals, hiring managers, and anyone relying on self-assessments in talent acquisition or people operations.

Support the show

Contact us at: totalsurveydesign@gmail.com

Find us online at: instagram.com/totalsurveydesign/

https://taplink.cc/totalsurveydesign

In this episode, I will be talking about open-ended questions. Along with closed-ended questions, open-ended questions are the other major type of question used in surveys. This episode covers the purpose of open-ended questions, their pros and cons, the different types of open-ended questions, several examples, and finally some tips and best practices for implementing them effectively in surveys.

The episode also includes an important ethical cautionary tale, as well as a warning about the dangers of over-relying on generative AI to analyze open-ended questions.

Support the show

Contact us at: totalsurveydesign@gmail.com

Find us online at: instagram.com/totalsurveydesign/

https://taplink.cc/totalsurveydesign

What happens when you are faced with a list of options but do not have a strong preference? In this episode, I explore how the simple order of a list can nudge a person's choice. I move beyond the idea of favorite flavors to look at the undecided voter and the hungry diner. I examine why being first on a ballot can sway an election by as much as 10 percent in some local races.

I discuss why these order effects are most dangerous in non-partisan elections where voters may not recognize every name on the list. Drawing on the extensive research of Jon Krosnick and others, I explain the difference between the Primacy Effect in written surveys and the Recency Effect in spoken lists. I provide practical solutions for researchers who want to ensure their data is a true reflection of the respondent rather than a fluke of the list order. I also cover randomization techniques and the best way to handle no opinion responses to maintain high data quality.

Support the show

Contact us at: totalsurveydesign@gmail.com

Find us online at: instagram.com/totalsurveydesign/

https://taplink.cc/totalsurveydesign