We’ve all been there; an email pops up in your inbox:
“Take our 5-minute survey for a chance to win an iPad!”
Five minutes? Free iPad? Tempting. But before you know it, you’re knee-deep in obscure questions like, “On a scale of ‘mildly amused’ to ‘existentially devastated,’ how would you rate our “on hold” music”?
Surveys are everywhere. Governments, businesses, and universities all love them. But are they truly valuable, or are they just glorified busywork that pays well for those running them? And most importantly, how do they make sure they’re gathering responses from the right people, and not just bored souls with nothing better to do?
The biggest challenge in survey design is ensuring that the responses reflect reality, not just the opinions of night-shift workers and people waiting for their pot noodle to process. If you’ve ever wondered why surveys seem to reflect the views of oddly enthusiastic demographics (“93% of people strongly agree that pineapple belongs on pizza!”), it’s probably because they didn’t ask the right people.
To avoid this, researchers use techniques like selecting participants at random to avoid only capturing the opinions of the most eager beavers. They divide people into groups (age, income, coffee consumption level) to ensure a good mix. I’m told they pre-set the number of responses needed from different demographics to ensure fair representation. In theory, this ensures that data reflects the real world rather than the opinions of Dave, 53, who enjoys filling out surveys in between tending to his exotic fish collection.
Anad yes, some people make a side hustle out of answering surveys. They speed through them just for the rewards, clicking random options like a caffeinated squirrel.
Good surveys have ways to catch the right participants, including screening questions to ensure you can’t participate in a survey about electrical cars if your only experience with these is listening to Mike moaning about having to run a cable from his third floor flat to charge his car which annoys Mrs Smith’s cat in the floor below…
Trying to maximise your chances of winning the mythical free iPad by completing the survey 55 times? If they run IP tracking and unique links, you’re as busted as Sarah who has claimed she’s 24 to enter a rewards based survey, but then later on0 clicked that she was at school in the 1980s.
One problem with surveys? The people you really need insights from are usually too busy doing important things, like working, parenting, or binge-watching an entire season of a show they “accidentally” started. Yup the ones who probably trashed your invite to win an iPad because it was an unsolicitored email and they thought it was a scam.
Researchers use incentives such as prize draws, charitable donations and gift cards to try to get us to respond, but even with these tricks, there’s still a risk that the only people responding are those who really care; hence why you get surveys claiming “87% of customers are passionate about better yogurt lid design.”
Even after all the effort to get the right people answering the right questions, survey data can still be skewed. That’s why researchers use weighting, a method that adjusts responses to reflect the actual population. If too many responses come from 18-year-olds and not enough from 60-year-olds, the data is balanced accordingly.
This helps prevent embarrassing statistical anomalies like “62% of people believe Flossing should be recognized as an Olympic sport.“
Bad surveys are just lazy data collection; designed poorly, sent to the wrong people, and analyzed in ways that make anyone with basic math skills cry. Good surveys take real effort and time; they use scientific methods to ensure their results aren’t just the opinion of the five people who had time to reply that day.
And let’s not forget the sneaky surveys that you probably don’t even notice; the digital world’s version of “Gotcha!” You think you’ve escaped a Zoom call unscathed, only to be ambushed by a cheery pop-up: “How was your meeting? 👍👎” Oh, so now I have to rate the awkward silence, Steve’s frozen face, and Janet’s accidental unmuting? These micro-surveys are the corporate world’s equivalent of a waiter asking “How’s your meal?” just as you’ve taken the world’s biggest bite. Worse, if you ignore them, you’re left wondering; do they know? Is someone, somewhere, watching a report that says, “User X refuses to engage with feedback. Mark them as ‘problematic.’” Next thing you know, you’re getting sound issues and mysteriously muted in meetings. Coincidence? Maybe.
Give a thumbs down? A bold move in the world of sneaky surveys, inviting feedback. But hit it, and suddenly, the system demands answers. “Oh no! What went wrong? Tell us more!” Oh, you want details? Fine. Bob spent 20 minutes explaining a spreadsheet that was already in the email. Someone’s dog barked the entire time. And I sat through it all, knowing full well this meeting could’ve been a memo. But do I really want to type all that? Nope. So I just click “Technical Issues” and move on with my life or just type “…” and let their AI figure that out.
The next time you see a genuine survey promising a chance to win an iPad for a five minute survey, just ask yourself: “yes five minutes, but on which planet?” Because here on Earth, that probably translates as 20 minutes, a burnt pizza and sometime later this year wondering… who won that iPad?.
Photo by Celpax
