Qualtrics is a dedicated survey system in SurveyTester. A project set to Qualtrics (or to Automatic) recognises a Qualtrics survey the moment the browser extension opens it: every page with its questions, answer options, matrix statements, sliders and text fields, the validation messages and the end of survey message. From there everything else in SurveyTester works as usual - Auto Answer, Auto Complete, SurveyBots, test cases, issues and the Pages & Routing overview.
What a Qualtrics survey looks like to a tester
A Qualtrics survey link usually looks like https://<organisation>.qualtrics.com/jfe/form/SV_.... The respondent
answers the questions of a block, clicks the arrow button (Next) and Qualtrics shows the next page. Before it moves
on, Qualtrics checks the answers - forced responses, content validation, totals - and shows a red message under the
question if something is wrong.
Qualtrics draws the same question in different ways, depending on the layout chosen for the survey (Flat, Modern or Classic): each produces different pages. SurveyTester reads all of them.
Why this is hard to test without SurveyTester
- One question, many layouts. A matrix can be a table, a carousel of cards or a star rating; a slider exists in three different designs. Each needs different clicks. SurveyTester recognises the design and answers it the way a respondent would.
- Questions that only work with the mouse. Rank order lists are dragged, sliders are pulled, dates are picked from a calendar. SurveyTester operates these controls so that Qualtrics accepts the answer.
- Totals and validation. Constant sum questions only move on when the numbers add up; required fields, number ranges and date formats are checked before the page continues. SurveyTester answers within these rules.
- Randomised answers. Qualtrics often shuffles choices and statements for every respondent. SurveyTester keeps the question's own identifiers, so answers stay comparable across runs.
- Sessions that resume. By default Qualtrics continues an unfinished response when the same browser opens the link again. See Qualtrics test runs not starting fresh.
Supported question types
| Qualtrics question type | How SurveyTester records it |
|---|---|
| Multiple choice, single answer (list, horizontal, dropdown list) | One choice is selected. |
| Multiple choice, multiple answer, also in several columns | Several choices are ticked. |
| Choices with a text entry box ("Other, please specify") | The choice and its text box are recorded together; the box gets a text. |
| Matrix table (Likert), single or multiple answers per statement | Every statement is answered with one or more scale points. |
| Matrix table with dropdowns or text boxes in the cells | Every cell gets a dropdown choice or a text. |
| Matrix statement with its own text box (e.g. "Other") | The text box is recorded with the statement and filled in. |
| Side by side | Every statement is answered in every column. |
| Carousel | The statements are answered card by card. |
| Star rating | Every row gets a number of stars. |
| Slider (all three Qualtrics slider designs) | Every slider is set to a value within its range. |
| Constant sum | Every item gets a number; the numbers add up to the required total (for example 100). |
| Rank order (drag and drop) | The order of the items is recorded; SurveyTester touches the list so that Qualtrics accepts the order as an answer. |
| MaxDiff (best and worst) | One best and one worst item are chosen in each set. |
| Text entry (single line, essay box) and form fields | A fitting text is entered in every field. |
| Number fields | A number is entered. |
| Date / calendar (single date or date range) | A date in the format the field expects is entered. |
| Descriptive text | Recorded as information with nothing to answer. |
Validation messages and the end page
- Validation messages. When Qualtrics refuses to continue, the red message under the question is stored as the error of that question - and of the text box it belongs to, for an "Other, please specify" box.
- End page. The end of survey message is recorded as the last step of the run. Complete, screenout and quota outcomes are configured as for any other survey system, with survey end page identifiers on the project. See Survey end pages.
Step by step
- Create the project. Choose Qualtrics as the survey system, or leave it at Automatic - the survey is recognised either way. Paste the survey link (the anonymous link or a test link).
- Start a test run from the project with the SurveyTester browser extension, or let SurveyBots run the survey. The SurveyTester bar appears above it, and the first page is recorded as step 1.
- Answer the survey by hand, or press Auto Answer and Auto Next to let SurveyTester fill the page and move on. Each page creates the next step.
- Check the result on the Pages & Routing page: every page appears with its questions, every question with its choices, statements and scale points.
Good to know
- Question names. SurveyTester uses the question's data export tag as shown in Qualtrics (for example
Q12), or the internal question ID (QID12) when the page does not show a tag. Keep the export tags stable between survey versions, so results stay comparable. - Start fresh. If a run starts in the middle of the survey, Qualtrics has resumed an earlier response. Start the test in a private window - see Using SurveyTester with incognito mode.
- Survey file upload. You can upload the Qualtrics survey file (
.qsf) before the first test run, so that the questions are known in advance. See Survey Metadata Upload. - Custom code. Questions rebuilt with your own JavaScript or HTML may look different from the standard Qualtrics questions. SurveyTester records their fields, but may not be able to answer them automatically. If you find a page that SurveyTester does not read correctly, please send it to us.