--- title: "Sandbox: check how a long brand list is spread over respondents" description: "Try the item exposure check on a demo survey that shows 25 of 200 brands per respondent: run SurveyBots and see SurveyTester judge the draw." url: https://www.surveytester.com/academy/en/sandbox/subset-item-exposure/ language: en type: how-to audiences: ["Survey Testers", "Project Managers", "Survey Authors"] published: 2026-10-18 updated: 2026-10-18 tags: ["sandbox", "demo", "subset", "item exposure", "brand list", "balanced draw", "surveybots", "dimensions"] --- # Sandbox: check how a long brand list is spread over respondents _Sandbox · SurveyTester Academy_ A brand list with 200 entries cannot be shown to a single respondent. The survey therefore shows every respondent a subset - here 25 brands - and the research team expects every brand to collect about the same number of answers over the fieldwork. That only works with a **balanced** (least-filled) draw: each respondent gets the brands that were shown least so far. If the survey just draws at random, some brands end up with three times as many answers as others, and nobody notices until the data is in. This sandbox lets you try SurveyTester's item exposure check on a demo survey that makes exactly this mistake. It takes about 10 minutes: the SurveyBots run in parallel, so a batch of 20 to 40 runs is done in a few minutes. The background on what is counted and how the verdict is formed is in [Testing questions that show a random subset of items](https://www.surveytester.com/academy/en/pages-and-routing/item-exposure-subset-questions). ## The demo survey `https://www.feedback-experts.com/mrIWeb/mrIWeb.dll?I.Project=SANDBOX_PARTIALDISTRIBUTION` Open the link in your browser first and click through it once - it takes a minute: | Page | What you see | What is behind it | | --- | --- | --- | | Age | Single choice | *Under 18* ends the interview as a screenout | | Awareness | 25 brands plus *None of these* | 25 brands drawn **at random** from a list of 200, *None of these* always shown | | Rating | A grid with 6 statements | 6 statements drawn at random from a list of 40 | | End | "Thank you for taking part" | | Every page looks perfectly fine. Whether brand 173 is ever shown, or how evenly the 200 brands are spread, only shows up when you count across many interviews - which is what SurveyTester does. ## Step 1: Create the project Open the project menu (☰) and choose **Create New Project**. Pick a folder, give the project a name such as "Sandbox brand list", and paste the link above as **Survey Link**. SurveyTester recognises the survey system (Dimensions) from the link. Choose English as default language and click **Create Project**. Everything else can stay as it is; the dialog is described in [Best practices: from a new project to sign-off](https://www.surveytester.com/academy/en/general-queries/project-best-practices). No rule or setting is needed: SurveyTester recognises subset questions from the test runs on its own. ## Step 2: Run a first batch of 20 SurveyBots Click **Test Survey**, choose **SurveyBot Testing** and set the number of test runs to 20. No test case is needed: random answers are fine. Start the job and wait until the runs show up in **Test Runs**. Runs that end as screenout at the age question are expected and do not count for the brand list - with random answers that is about one run in four. ## Step 3: Look at the counts in Pages & Routing Open **Pages & Routing** and reload the page once the runs are finished. In the **Expectations** column you now see: - **25 of 200** on the Awareness question - SurveyTester found that every interview shows 25 brands out of a list of 200, with *None of these* always shown and therefore left out of the count; - **6 of 40** on the Rating grid, for its statement rows. ![Placeholder: screenshot of the Pages & Routing table with the buttons "25 of 200" on Awareness and "6 of 40" on Rating in the Expectations column, both dashed grey](https://www.surveytester.com/academy/_img/sandbox/subset-item-exposure/en/sandbox-subset-buttons.png "w=44rem | The two subset questions, recognised from the test runs") Click **25 of 200** to open the **Item exposure** dialog. After 20 runs the button is still dashed grey and the dialog says *No verdict yet*: every brand has been shown only about twice so far, which is not enough to tell a balanced draw from a random one. The evidence line says how many runs are still needed, and the item table already shows the counts per brand - look at how far apart they are. ![Placeholder: screenshot of the Item exposure dialog after 20 runs - No verdict yet, evidence line asking for more runs, item table with counts per brand](https://www.surveytester.com/academy/_img/sandbox/subset-item-exposure/en/sandbox-subset-no-verdict.png "w=44rem | After 20 runs: counts visible, verdict still open") ## Step 4: Run 40 more SurveyBots and read the verdict Start another SurveyBot job with 40 runs. Once about 40 runs have reached the brand list, every brand has been shown about five times on average, and the check judges. If the evidence line still asks for more runs, start a few more. Reload **Pages & Routing**: the button turns orange and the dialog says: - **Not OK - Random draw, not balanced.** The counts spread exactly as much as a random draw produces. With 25 of 200 brands, some brands have been shown once or twice, others ten times or more, although every brand should sit at about five; - an **evenness score** of roughly 60 to 70 (a balanced draw stays above 90); - in the facts, the band chance alone produces - so you can see that the spread is not bad luck, but how a random draw behaves. ![Placeholder: screenshot of the Item exposure dialog after about 40 interviews - verdict Not OK, Random draw not balanced, evenness score about 65, item bars of very different length](https://www.surveytester.com/academy/_img/sandbox/subset-item-exposure/en/sandbox-subset-random.png "w=44rem | After about 40 interviews: the survey leaves the exposure to chance") The Rating grid gets the same verdict at about the same time: its six statements are drawn at random too. Each deviation is also an AutoDetect finding *Item exposure not balanced* under the check *Item Exposure* in **Issues**, ready to be approved into an issue for the survey author. > [!NOTE] > The check *Every item of the list is shown* may still wait at this point: with a random draw, chance > alone can leave one brand unseen for a while. It is judged after about 60 interviews that reached the brand list. > Run another 20 to 30 SurveyBots if you want to see it. ## What a correct survey would show A survey with a balanced draw - Dimensions with sample management or a show-count table, Decipher's balanced `shuffle`, Qualtrics' "evenly present elements" - keeps every brand at five or six shows after 40 interviews. The dialog then says **Very likely OK** with an evenness score above 90. If your study really intends a random draw, state it as a rule in the question's **Expectations** dialog (*Subset per respondent*, draw *random*): the random draw is then accepted. Both are described in [Testing questions that show a random subset of items](https://www.surveytester.com/academy/en/pages-and-routing/item-exposure-subset-questions). ## What you have practised - SurveyTester recognises questions that show only part of a list from the test runs alone, with no setup. - Whether a long list is spread evenly is invisible in a single interview and becomes visible after a few dozen SurveyBot runs. - The dialog tells a balanced draw from a random or biased one, and says when the evidence is sufficient. Use the same steps on your next study with a long brand or statement list: run enough SurveyBots before launch for the evidence line to say *sufficient*, open the **n of N** button and hand a random or biased draw to the survey author before fieldwork starts.