![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Entry tags:
polls, surveys, quizzes, and questionnaires
Sometimes little online quizzes and things can be fun--things that tell you which character in XX show/book you are, or whatever. But as you've probably all experienced taking those quizzes, they reveal as much or more about the assumptions of the quiz creator as they do about you.
That's one of my huge--sometimes insurmountable--problems with health questionnaires or political polls, too. (With political polls, another huge problem I have is who the hell is actually answering them. I *never* respond to political polls--largely because of problems related to the questions, but also because I don't answer spam phone calls etc.--so what person has the time and inclination to answer them? How biased are all polls, when they only draw on the pool of people willing to respond to polls?)
One basic thing that all quizzes, polls, and questionnaires often do wrong is that when they offer you responses, they also supply reasons for the response. For example:
Q. Do you pick up roadside litter?
A1. Yes, always; I can't stand that jerks are always leaving their trash around.
A2. Yes, sometimes, if I'm not in a hurry
A3. No, never--why should I pick up after other people?
All these answers have rationales that may not be the respondents' rationale at all. For instance, maybe you never pick up trash because you're immunocompromised and don't want to get sick. Or maybe you always pick up trash because you're interested in what things people discard. Or maybe you sometimes do, but it's not so much whether you're in a hurry or not but whether you have a free hand. And so on. (Never mind that the answers don't allow for certain meaningful other answers, such as "Yes, but only cans and bottles that I can collect the deposit on.")
If the rationale in an answer isn't your rationale, then you're forced to either choose that answer, even though it misrepresents your thinking, or not answer at all.
For that reason, I prefer questions that don't put rationales in the response choices .... But of course, the quiz creators would still be assuming the rationales, only their assumptions are hidden from you.
All of this is to say, always, always be skeptical of responses to questionnaires and polls. You don't know what kind of crap thinking went into their creation, how they were worded, and so on.
That's one of my huge--sometimes insurmountable--problems with health questionnaires or political polls, too. (With political polls, another huge problem I have is who the hell is actually answering them. I *never* respond to political polls--largely because of problems related to the questions, but also because I don't answer spam phone calls etc.--so what person has the time and inclination to answer them? How biased are all polls, when they only draw on the pool of people willing to respond to polls?)
One basic thing that all quizzes, polls, and questionnaires often do wrong is that when they offer you responses, they also supply reasons for the response. For example:
Q. Do you pick up roadside litter?
A1. Yes, always; I can't stand that jerks are always leaving their trash around.
A2. Yes, sometimes, if I'm not in a hurry
A3. No, never--why should I pick up after other people?
All these answers have rationales that may not be the respondents' rationale at all. For instance, maybe you never pick up trash because you're immunocompromised and don't want to get sick. Or maybe you always pick up trash because you're interested in what things people discard. Or maybe you sometimes do, but it's not so much whether you're in a hurry or not but whether you have a free hand. And so on. (Never mind that the answers don't allow for certain meaningful other answers, such as "Yes, but only cans and bottles that I can collect the deposit on.")
If the rationale in an answer isn't your rationale, then you're forced to either choose that answer, even though it misrepresents your thinking, or not answer at all.
For that reason, I prefer questions that don't put rationales in the response choices .... But of course, the quiz creators would still be assuming the rationales, only their assumptions are hidden from you.
All of this is to say, always, always be skeptical of responses to questionnaires and polls. You don't know what kind of crap thinking went into their creation, how they were worded, and so on.
no subject
So at the end of the day, what you had to do if you wanted to let everyone give them honest and detailed reasoning - and have more accurate results - is to create other questions that people can actually respond to in free-form. But that means more work afterwards, and you'd have to sift through all the responses and create categories for them to be able to analyze them statistically, and a lot of people are just too lazy to do that, I suppose.
no subject
This, to me, is key. Multiple-response questions yield easily quantifiable data, but often, for all the reasons we're saying, it's garbage data, or at the very least, very flawed. If you want to understand how people feel about a thing, then you need to **listen to them**. But this takes massive amounts of time (and of course still depends on the listeners to be capable of understanding what they're hearing, when often people are so lost in their own ideas that they aren't good at hearing something novel or unexpected).
I think a whole lot of how we live today is based on trying to escape true costs, and that carries over into human interaction. To really understand issues in a holistic way, you need to let people talk in an open-ended way, but the machine of daily life can't accommodate the cost of that. To our regret and detriment.
no subject
no subject