Table of Contents
Somewhere between 5% and 25% of your survey responses are worthless. Bots, professional speedrunners, and humans watching Netflix with your survey in a second tab all leave the same residue in your dataset: answers that look like data and mean nothing. Attention checks are how you find them.
But attention checks are easy to write badly. A trick question that snags honest-but-tired respondents does not clean your data; it deletes good responses and breeds resentment. This guide covers how to write checks that separate inattention from imperfection, how many to use, and what to do when someone fails. If you want ready-made checks, the attention check generator has 20+ you can copy directly. For the case that attention checks matter at all, see the importance of attention checks in survey research.
The four types of attention check that work
1. Instructed response items
The classic. A question that tells the respondent exactly what to do:
"To show you are reading carefully, please select 'Somewhat disagree' for this item."
Anyone answering honestly passes in two seconds. Anyone straightlining or botting fails. Format it exactly like the surrounding scale items so it does not visually stand out; the point is that only readers notice it.
2. Consistency pairs
Ask the same thing twice, differently phrased, at a distance:
Early in the survey: "I enjoy working in teams."
Later: "I prefer working alone."
A respondent who strongly agrees with both is not reading. Consistency pairs double as reverse-coded items in a multi-item scale, so you can often get this check for free from a well-designed instrument. Allow some wiggle room in scoring; agreeing mildly with both is human, agreeing strongly with both is a flag.
3. Bogus items
Statements that have one true answer for everyone:
"I have never used the internet." (You are taking an online survey.)
"I have visited every country in the world."
Keep bogus items literally impossible rather than merely unlikely, and avoid anything a non-native speaker could misread. The goal is to catch random clicking, not vocabulary gaps.
4. Open-ended micro-checks
A short free-text question with a verifiable answer:
"In one sentence, what was the scenario you just read about?"
These are the strongest bot detectors you can write yourself, because generating a relevant sentence requires actually processing the content. They also catch AI-assisted responding better than multiple-choice checks; more on that in how to detect AI-generated survey responses.
Checks that punish honest respondents
Avoid trivia ("what color was the third image?"), memory tests disguised as attention checks, and instructions buried inside long paragraphs nobody was told to memorize. If a careful respondent reading at normal speed can fail it, it is a bad check. You want a floor of attention, not a ceiling.
How many checks, and where
The evidence-backed sweet spot for a typical 10 to 20 minute survey is two to three checks, spaced out:
Attention check placement plan
| Survey length | Checks | Placement |
|---|---|---|
| Under 5 minutes | 1 | Middle |
| 5–15 minutes | 2 | One-third and two-thirds through |
| 15+ minutes | 3 | Spread evenly, none in the first minute |
Never put a check in the first few questions (attention is naturally high there; you learn nothing) and avoid stacking two checks together (failing both is one event, not two). Mix types: an instructed response plus a consistency pair covers more failure modes than two instructed responses.
Decide your failure policy before fieldwork
The fairest policies are boring and pre-registered:
- One failure: flag the response, keep it, examine it during cleaning alongside other signals (speed, straightlining).
- Two or more failures: exclude, and report the exclusion count in your write-up.
- Never silently delete and never decide the threshold after seeing the results. Choosing an exclusion rule that improves your findings is p-hacking with extra steps.
Speed is the natural companion signal. A response at one-third of the median completion time with one failed check is a far stronger exclusion case than either signal alone. Timing checks and duplicate detection are also exactly the kind of thing that should run automatically; that is the job of SurveySwap Protect, built to screen out bots, duplicates, and low-effort answers before they reach your dataset.
What researchers typically catch
Sample quality changes everything
Attention check failure rates are as much a fact about your sample as about your survey. Anonymous social media links fail at multiples of the rate of engaged panels, because the incentive to speed through is higher and accountability is zero. Reciprocity-based samples sit at the favorable end: on SurveySwap's exchange, respondents are researchers themselves who need their own data to be taken seriously, which changes how they treat yours.
Frequently asked questions
What is a good attention check for a survey?
An instructed response item ("please select 'Agree' for this question") formatted like the surrounding questions is the reliable default. Pair it with a consistency check or an open-ended micro-check for broader coverage. The attention check generator has copyable examples of each type.
How many attention checks should a survey have?
Two to three for a typical survey, spaced through the middle. One is enough below five minutes. More than three adds respondent fatigue faster than it adds detection.
Should I tell respondents attention checks are present?
A brief note that the survey contains quality checks is fair, slightly improves attention, and does not help cheaters, since they still have to find the checks. Do not reveal where or what they are.
Can I reject participants who fail one attention check?
Best practice is to flag on one failure and exclude on two or more, combined with other quality signals like completion speed. Whatever rule you choose, set it before data collection and report it.
Attention checks are one tool of several. Browse the full free research tools collection.
