![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Entry tags:
polls, surveys, quizzes, and questionnaires
Sometimes little online quizzes and things can be fun--things that tell you which character in XX show/book you are, or whatever. But as you've probably all experienced taking those quizzes, they reveal as much or more about the assumptions of the quiz creator as they do about you.
That's one of my huge--sometimes insurmountable--problems with health questionnaires or political polls, too. (With political polls, another huge problem I have is who the hell is actually answering them. I *never* respond to political polls--largely because of problems related to the questions, but also because I don't answer spam phone calls etc.--so what person has the time and inclination to answer them? How biased are all polls, when they only draw on the pool of people willing to respond to polls?)
One basic thing that all quizzes, polls, and questionnaires often do wrong is that when they offer you responses, they also supply reasons for the response. For example:
Q. Do you pick up roadside litter?
A1. Yes, always; I can't stand that jerks are always leaving their trash around.
A2. Yes, sometimes, if I'm not in a hurry
A3. No, never--why should I pick up after other people?
All these answers have rationales that may not be the respondents' rationale at all. For instance, maybe you never pick up trash because you're immunocompromised and don't want to get sick. Or maybe you always pick up trash because you're interested in what things people discard. Or maybe you sometimes do, but it's not so much whether you're in a hurry or not but whether you have a free hand. And so on. (Never mind that the answers don't allow for certain meaningful other answers, such as "Yes, but only cans and bottles that I can collect the deposit on.")
If the rationale in an answer isn't your rationale, then you're forced to either choose that answer, even though it misrepresents your thinking, or not answer at all.
For that reason, I prefer questions that don't put rationales in the response choices .... But of course, the quiz creators would still be assuming the rationales, only their assumptions are hidden from you.
All of this is to say, always, always be skeptical of responses to questionnaires and polls. You don't know what kind of crap thinking went into their creation, how they were worded, and so on.
That's one of my huge--sometimes insurmountable--problems with health questionnaires or political polls, too. (With political polls, another huge problem I have is who the hell is actually answering them. I *never* respond to political polls--largely because of problems related to the questions, but also because I don't answer spam phone calls etc.--so what person has the time and inclination to answer them? How biased are all polls, when they only draw on the pool of people willing to respond to polls?)
One basic thing that all quizzes, polls, and questionnaires often do wrong is that when they offer you responses, they also supply reasons for the response. For example:
Q. Do you pick up roadside litter?
A1. Yes, always; I can't stand that jerks are always leaving their trash around.
A2. Yes, sometimes, if I'm not in a hurry
A3. No, never--why should I pick up after other people?
All these answers have rationales that may not be the respondents' rationale at all. For instance, maybe you never pick up trash because you're immunocompromised and don't want to get sick. Or maybe you always pick up trash because you're interested in what things people discard. Or maybe you sometimes do, but it's not so much whether you're in a hurry or not but whether you have a free hand. And so on. (Never mind that the answers don't allow for certain meaningful other answers, such as "Yes, but only cans and bottles that I can collect the deposit on.")
If the rationale in an answer isn't your rationale, then you're forced to either choose that answer, even though it misrepresents your thinking, or not answer at all.
For that reason, I prefer questions that don't put rationales in the response choices .... But of course, the quiz creators would still be assuming the rationales, only their assumptions are hidden from you.
All of this is to say, always, always be skeptical of responses to questionnaires and polls. You don't know what kind of crap thinking went into their creation, how they were worded, and so on.
no subject
"No, I don't pick up rubbish - bending over aggravates my back/hip pain for days" is not an option...
Around 1996/1997/1998/1999 I once had a market researcher come to my door with a clipboard and ask how many hours of TV I watched - zero, I said.
They flat out refused to believe me - partly because zero was not an option on their form.
Finally I had to say "Look, MY HOUSEMATE MOVED OUT AND TOOK THE TV. THERE IS LITERALLY NO TV IN THE HOUSE. THEREFORE IT IS IMPOSSIBLE FOR ME TO WATCH MORE THAN ZERO HOURS OF TV"
no subject
no subject
The number of surveys that ask how many standard alcoholic drinks you have per week and don't let you choose ZERO is... a lot.
no subject
no subject
a) Australians often drink alcohol
b) the surveys are all of over-18s, and 18 = legal drinking age in Australia
but we do have a not-insignificant amount of people in Australia who don't drink because of religious reasons like Islam or Baháʼí
not to mention people who don't drink because
a) health issues - including pregnancy, for fuck's sake!
b) it clashes with medications like antidepressants
no subject
My pet hate is the survey that forces me to select a neutral 'don't know / don't care' option to express 'I have SO MANY opinions about this...'
no subject
no subject
Remember when people were talking about George Lakoff's Don't Think of an Elephant? It seems now entirely to have vanished from active memory.
*rages uselessly*
no subject
no subject
I was so excited! I had been mystified about the falling into framing that is absolutely neither absolute nor inevitable! Someone people listened to was saying it!
(He also suggested Dems putting in the sort of volunteer, communication and donation groundwork that Republicans had been doing all along.)
During the period in which people liked to mention the title of the book, I got highatted several times of people telling me how George Lakoff would dismiss my framing.
And now he's seemed forgotten for at least five years.
no subject
Sometimes valuable message don't "stick"
no subject
no subject
no subject
no subject
no subject
no subject
Personality-related questionnaires, whether serious or frivolous, always fall down by failing to make the distinction between personality traits and actual behaviour. They are also seem to be always designed for and by Americans.
no subject
no subject
Context is just as important a determinant of action as personality. The enemy could just have cut off the castle's water-supply. Immediate surrender, whatever your personality (see Fall of Singapore Feb 1942).
no subject
no subject
no subject
no subject
no subject
Oh, yes. This. It is so easy to twist the most seemingly benign info into a Thing It Is Not to support the Thing They Want It To Be.
o.O
no subject
no subject
This reminds me of something I heard on an unschooling podcast, where a mother shared how she was able to start having meaningful and trust-building conversations and relationships when she no longer framed everything as "How can I get my child to come around to my way of thinking?"
no subject
With very little children, there are some things that I don't think are open for discussion, things that, as a parent, I think should just happen, but as soon as a child is of an age to begin to talk and understand things, then yeah: make it be real.
no subject
no subject
no subject
So at the end of the day, what you had to do if you wanted to let everyone give them honest and detailed reasoning - and have more accurate results - is to create other questions that people can actually respond to in free-form. But that means more work afterwards, and you'd have to sift through all the responses and create categories for them to be able to analyze them statistically, and a lot of people are just too lazy to do that, I suppose.
no subject
This, to me, is key. Multiple-response questions yield easily quantifiable data, but often, for all the reasons we're saying, it's garbage data, or at the very least, very flawed. If you want to understand how people feel about a thing, then you need to **listen to them**. But this takes massive amounts of time (and of course still depends on the listeners to be capable of understanding what they're hearing, when often people are so lost in their own ideas that they aren't good at hearing something novel or unexpected).
I think a whole lot of how we live today is based on trying to escape true costs, and that carries over into human interaction. To really understand issues in a holistic way, you need to let people talk in an open-ended way, but the machine of daily life can't accommodate the cost of that. To our regret and detriment.
no subject
no subject