Testing Alchemer (SurveyGizmo) surveys

Alchemer, formerly SurveyGizmo, is a dedicated survey system in SurveyTester. In the survey system list it still carries its old name: a project set to SurveyGizmo (or to Automatic) recognises an Alchemer survey the moment the browser extension opens it - every page with its questions, answer options, table rows and text fields, and the end page. From there everything else in SurveyTester works as usual - Auto Answer, Auto Complete, SurveyBots, test cases, issues and the Pages & Routing overview.

What an Alchemer survey looks like to a tester

Alchemer is a hosted survey platform. A survey link usually points to https://survey.alchemer.com/s3/<id>/<name> (older surveys use surveygizmo.com, European accounts alchemer.eu); surveys can also run on an account's own domain. Every page is a classic web page: the respondent answers and clicks Next, on the last page Submit. Alchemer checks the answers, and if something is wrong it shows the same page again with a red message on top and under each question that needs attention.

Why this is hard to test without SurveyTester

• Logic that hides questions. Alchemer hides questions with page logic and keeps them in the page. SurveyTester skips hidden questions, so nothing is answered that a respondent would never see.
• Error pages that repeat the page. After an error Alchemer shows the page again, with the accepted answers still filled in. SurveyTester keeps those answers and only answers the questions that were rejected.
• Rules hidden in the survey. Number ranges, character limits, percentage fields and lists of allowed values for a text box are part of the survey's settings, not of the visible page. SurveyTester reads them and answers within them.
• Several steps on one page. A MaxDiff question shows its sets one after the other on the same page. SurveyTester answers each set and moves on to the next one before the page is submitted.

Supported question types

Alchemer question type	How SurveyTester records it
Radio buttons	One option is chosen.
Checkboxes, with exclusive options	Several options are ticked; an exclusive option ("None of these") is ticked alone.
Option with "Other" text field	The option and its text field are recorded together.
Dropdown menu	One entry of the menu is chosen.
Image choice	One picture is chosen.
Net Promoter Score	One value from 0 to 10 is chosen.
Textbox, essay	A fitting text is entered, respecting the minimum and maximum number of characters.
Textbox with autocomplete list	One of the listed values is typed in.
E-mail, number, percent and date fields	An e-mail address, a number in the allowed range, a percentage or a date is entered.
Slider	The slider is set to a value on its scale.
Radio button table, checkbox table	Every row is answered with one column or with several ticks.
Dropdown menu table, textbox table	Every cell gets an entry from its menu, or a text or number.
MaxDiff	Every set gets one best and one worst item.
Contact form and custom groups	Every field of the group is answered as a question of its own: text, e-mail, number or dropdown.
Text and media elements	Recorded as information without an answer.
File upload	Recorded but left alone: files cannot be uploaded automatically.

Errors and end page

• Error messages. The red banner at the top of the page is stored as the page error, the message under a question as the error of that question. Both show up in the test run and in issues.
• End page. The thank-you page - a page without Next and Submit - and Alchemer's terminate pages are recorded as the last step of the run. Complete, screenout and quota outcomes are configured as for any other survey system, with survey end page identifiers on the project. See Survey end pages.

Step by step

1. Create the project. Choose SurveyGizmo as the survey system, or leave it at Automatic - the survey is recognised either way. Paste the survey link.
2. Start a test run from the project with the SurveyTester browser extension, or let SurveyBots run the survey. The SurveyTester bar appears above it, and the first page is recorded as step 1.
3. Answer the survey by hand, or press Auto Answer and Auto Next to let SurveyTester fill the page and move on. Each page creates the next step.
4. Check the result on the Pages & Routing page: every page appears with its questions, every question with its answer options, rows and columns.

Good to know

• Names. Pages carry the page title you gave them in Alchemer, or p and the page id when there is no title. Questions are named q and Alchemer's question id (q12), so they stay the same across runs and languages.
• Interactive widgets. Drag-and-drop rankings, card sorts and other interactive question designs are recorded with their fields, but may not be answered automatically.
• Custom code. Surveys with their own JavaScript or heavily customised themes may not be read completely. If you find a page that SurveyTester does not read correctly, please send it to us.