Every survey starts as a document: the questionnaire the researcher wrote, with questions, answer lists, routing, termination rules and quotas. Programming turns that document into a survey, and testing means checking that the survey does what the document says. SurveyTester reads the questionnaire itself, so the document becomes the reference for testcases, for verification and for text comparison - with no retyping.
What the questionnaire tab does
The Questionnaire tab of a project takes the document as Word (.docx), Excel (.xlsx) or PDF. SurveyTester reads it with AI into a structured model: questions with codes and texts, answer options and grid rows, named lists, routing ("ask if"), termination rules, quotas and constructs. The layout does not matter: a classic questionnaire with codes at the line start, an agency template with scaffold lines, a tracker with bracket notes or a spreadsheet are all read. Hidden variables and information pages are recognised as well.
Every question of the model is shown with the original document fragment, the recognised structure and, once test runs exist, the screenshot of the survey page side by side. The whole document is viewable in the browser.
Why this is hard without SurveyTester
- Two sources of truth. The document is what the client signed off; the survey is what respondents see. Checking one against the other by hand means reading every question twice.
- Routing hides in notes. "Ask if Q3 = 1 or 2", "terminate if under 18", "show the brands selected at Q5" - rules scattered over programming notes, hard to test exhaustively by clicking.
- Testcases take time. Writing one testcase per routing branch and one per screenout rule is the work nobody has time for before launch.
How it works
1. Upload and analysis
Drop the document on the Questionnaire tab or attach it in the new-project dialog. The analysis starts right away (switch Analyse right after the upload off to start it later). The status moves from Uploaded to Analysing to Analysed; the progress line shows the current AI call. Every reading is confirmed by several independent passes, a repair pass picks up lines the extraction missed, and option labels are verified against the document text. The reasoning effort configured for the AI engine applies to the analysis.
The Overview tab lists title, language, legend, sections, lists, quotas and constructs with warnings; the Questions tab every question with code, type, text, options, rows, "ask if", termination and scripter notes.
2. Testcases and bots from the document
Generate testcases derives personas from the model: one Complete persona per quota cell and routing branch, and one Screenout persona per termination rule. Generate & run bots starts SurveyBots with them at once - this works before the survey has ever been tested, because the bots answer live per page from the persona description. The testcases are listed in the Testcases tab and in the project's testcase list.
3. Match & verify
Once test runs exist, Match & verify maps the document questions to the detected survey questions (grids programmed as a loop of questions included) and checks the survey against the document:
| Checked | Examples |
|---|---|
| Structure | missing or extra options, exclusive flags, selection limits, question type |
| Wording | question and option texts that differ from the document |
| Per test run | routing ("ask if"), termination rules, piping |
| Programming rules | randomisation, piped lists, subsets per respondent - handed to the question expectations |
Discrepancies appear in the Findings tab with a severity. A finding can be declined (it stays declined when the next verification reproduces it) or turned into an issue; a summary tells you what to do next. The verification runs again automatically after every bot job.
4. Text comparison
Compare texts opens the translation view with the document as a column: every question, option and grid row text of the survey is compared word by word with the document, and differences can be reported as issues. The former External Files upload of the translation view has moved here; files uploaded there earlier keep working until their document is analysed. Uploaded documents are also listed in Docs & Info for download.
Step by step
- Create the project with the survey link and upload the questionnaire in the Questionnaire tab (or attach it in the new-project dialog).
- Wait for the status Analysed and skim the Overview for warnings - a list the analysis could not read, a condition that references an unknown code.
- Generate & run bots: the personas cover every routing branch and every screenout.
- When the bots are through, Match & verify and work the Findings: decline what is intended, turn the rest into issues for the programmer.
- After the fix, run the bots again; the verification repeats itself and withdrawn findings disappear.
- Use Compare texts for the final wording check against the document.
Notes
- The analysis uses the company's AI engine; an administrator can switch it off for questionnaires, see AI Administration. Usage is metered per project.
- A document can be re-analysed after an edit (Re-analyse); mappings and declined findings are kept where the question codes still match.
- Legacy .doc files are converted; scanned PDFs without a text layer cannot be read.
Key takeaway
The questionnaire document is the reference for the whole test: SurveyTester reads it once, derives the testcases, verifies the programmed survey against it after every bot run, and compares the texts - so the survey that goes live is the one the document describes.