LimeSurvey is a dedicated survey system in SurveyTester. A project set to LimeSurvey (or to Automatic) recognises a LimeSurvey survey the moment the browser extension opens it - on LimeSurvey Cloud and on your own server alike: the welcome page, every question group with its questions, answer options, array rows and text fields, and the end page. From there everything else in SurveyTester works as usual - Auto Answer, Auto Complete, SurveyBots, test cases, issues and the Pages & Routing overview.
What a LimeSurvey survey looks like to a tester
LimeSurvey is open-source survey software. Many organisations run it on their own server, others use the hosted
LimeSurvey Cloud. A survey link usually looks like https://<server>/index.php/<survey id>?lang=en. Depending on the
survey's format, a page shows one question group, one single question or the whole questionnaire. The respondent
answers and clicks Next; LimeSurvey checks the answers and either shows the next page or marks the questions that
are missing or invalid. Before the first group there may be a welcome page, a data policy to accept, an access code or
a CAPTCHA.
Why this is hard to test without SurveyTester
- Many versions and themes. LimeSurvey surveys run on very different versions (3, 5 and 6 are all common) and themes. The same question type looks different in each of them; SurveyTester reads them the same way.
- Hidden conditions. LimeSurvey keeps questions whose condition is not met in the page, just invisible. SurveyTester skips them, so they are neither answered nor reported as open questions.
- Rules shown as tips. Number ranges, minimum and maximum answers and date formats appear as small tips under the question and turn red once a rule is broken. SurveyTester reads them and answers within the limits.
- Pop-ups between pages. When a page is not complete, LimeSurvey opens an alert box or asks whether to continue without answering. SurveyTester closes the box and answers the missing question instead of skipping it.
Supported question types
| LimeSurvey question type | How SurveyTester records it |
|---|---|
| List (radio), 5 point choice, Yes/No, Gender | One option is chosen. |
| List (dropdown) | One entry of the list is chosen; the "Please choose" placeholder is never picked. |
| List with comment | One option is chosen and the comment field is filled in. |
| Multiple choice, also with comments and exclusive options | Several options are ticked; an exclusive option is ticked alone, comment fields are filled in. |
| "Other" option with a text field | The option and its text field are recorded together. |
| Image select (single or multiple) | The pictures are recorded as options, named after their image. |
| Short, long and huge free text | A fitting text is entered. |
| Numerical input, with minimum and maximum | A number within the allowed range is entered. |
| Multiple numerical input, multiple short text | Every field of the question gets its own number or text. |
| Date | A date in the survey's date format and within the allowed dates is entered. |
| Array (flexible labels, 5 point, 10 point, Increase/Same/Decrease, Yes/No/Uncertain) | Every row is answered with one scale point. |
| Array by column | The columns are recorded as the items, so every column gets its answer. |
| Array dual scale | Every row gets an answer on both scales. |
| Array (numbers) and Array (texts) | Every cell gets a number from its dropdown or checkbox, or a text. |
| Ranking (classic and advanced) | Items are placed on ranks one after the other. |
| Slider (multiple numerical input as sliders) | Every slider is set to a value on its scale. |
| Single checkbox | Recorded as a yes/no question: the box is ticked or left empty. |
| Text display | Recorded as information without an answer. |
| File upload | Recorded but left alone: files cannot be uploaded automatically. |
| Language switch | Recorded; the language of the test run is kept. |
Welcome page, errors and end page
- Welcome page and data policy. The welcome page is recorded as the first step. If the survey asks to accept its data policy, SurveyTester ticks the box so the survey can start.
- Access code and CAPTCHA. Closed surveys ask for an access code (token), some surveys for a CAPTCHA. SurveyTester never fills these in: the test run waits until you have entered the code or solved the puzzle yourself, and then continues with the first question group. A public registration form is recorded as its own page.
- Error messages. Messages for missing or invalid answers are stored as errors of the question they belong to; messages for the whole page are stored as page errors. Both show up in the test run and in issues.
- End page. LimeSurvey's completion page is recorded as the last step of the run. Complete, screenout and quota outcomes are configured as for any other survey system, with survey end page identifiers on the project. See Survey end pages.
Step by step
- Create the project. Choose LimeSurvey as the survey system, or leave it at Automatic - the survey is recognised either way. Paste the survey link, with the language parameter if the survey has several languages.
- Start a test run from the project with the SurveyTester browser extension, or let SurveyBots run the survey. The SurveyTester bar appears above it, and the first page is recorded as step 1.
- Answer the survey by hand, or press Auto Answer and Auto Next to let SurveyTester fill the page and move on. Each page creates the next step.
- Check the result on the Pages & Routing page: every page appears with its questions, every question with its answer options, rows and scale points.
Good to know
- Names. Pages carry the name of their question group, questions their LimeSurvey question code (for example
Q002) where the theme shows it. Answer values are LimeSurvey's own answer codes, so test cases read like the survey's data export. - "No answer" options. Optional questions in LimeSurvey offer a "No answer" choice. SurveyTester does not count it as an answer option; leaving a question unanswered is something you set up in a test case.
- Custom themes. Surveys with heavily customised themes or their own JavaScript widgets may not be read completely. If you find a page that SurveyTester does not read correctly, please send it to us.