QA does not end when an issue is reported. The programmers fix it, and then somebody has to look again: is the typo really gone, does the question rotate now, is the old wording replaced everywhere? With many issues and several languages, this "look again" is a large part of the testing work.
SurveyTester does most of it for you. When an issue is marked for a re-check, SurveyTester waits for the next capture of the issue's page, checks the issue again in the same way it was found, and tells you the result: looks fixed, still present or please check manually. A person always has the last word. The result never closes or re-opens an issue on its own.
Why this is hard without SurveyTester
- Re-testing is repetitive. Every fixed issue means opening the survey, reaching the right page with the right answers, and comparing it with what was reported. For a project with fifty issues in three languages, that is a day of clicking.
- The page is hard to reach. Many issues sit deep in the survey, behind screeners, quotas and routing. Getting back to the exact page often means repeating the whole interview.
- "Fixed" is a claim, not a fact. A status change by the programmer says the fix was made, not that it works. Without a re-test, fixed issues reach the client unchecked.
- Some problems only show over many interviews. A missing rotation or an unbalanced MaxDiff design cannot be seen on one page. Old test runs keep proving the problem, even after the fix.
How it works
- The issue is marked for a re-check. Usually the programmer sets a status such as Re-check after the fix. SurveyTester notices the mark and opens a re-check request.
- SurveyTester waits for a fresh capture of the page. Any test run that shows the issue's page in the issue's language counts: a manual test, a test link or a SurveyBot. If nobody tests the page again, you can start a SurveyBot replay that repeats the original interview up to that page. For manually reported issues a person has to look, so SurveyTester offers a manual repeat of the test run instead.
- The issue is checked again. How depends on where the issue came from (see the table below).
- You are notified. The result is added to the issue history and sent like any other issue change, by e-mail and in the notifications.
- You decide. Confirm fixed closes the issue with your project's closing status. Still broken sends it back to the programmers. Testers can also decide while they look at the page: the browser extension asks Is this solved? at the issue (see Answering re-checks while testing).
| The issue came from... | How SurveyTester re-checks it | Result |
|---|---|---|
| An accepted Auto-Detect finding on a page (spelling, wrong language, unresolved pipe-in, visible HTML, structure checks) | Runs the same check again on the new capture | Looks fixed when the problem is gone and the question is still on the page; still present when the same problem is found again |
| A text change (the issue has an old and a new text) | Looks for both texts on the new capture | Looks fixed when the new text is shown and the old one is gone; still present when the old text is still there |
| A project-wide finding (rotation, item dependency, anchored items, MaxDiff balance, item exposure) | Runs the same check, but only over the test runs started after the re-check request | Looks fixed when the new runs meet the expectation; still present when they show the problem again |
| A manually reported issue | Shows you the new capture next to the original screenshot. If your company allows it, an AI suggests whether the problem is still visible | Always please check manually - the AI suggestion helps, the decision is yours |
Step by step
1. Set up the re-check status (administrators)
The re-check works like closing: a classification value can set it. Open Administration → Project Types, select the project type and the status field, and tick Needs Re-check next to the value your programmers use after a fix (for example Status = Re-check). From now on, choosing this value marks the issue for a re-check. Choosing a closing value, or any other value, removes the mark.
If a project type has no such value, the Re-check checkbox in the issue list and the Re-check switch in the issue dialog can be set by hand.
2. Mark issues for a re-check
The programmer sets the re-check status on the fixed issues, one by one or with Bulk Edit. The issue list shows the mark in the Re-check column. Add the column with the column selector if you do not see it.
A closed issue never needs a re-check: closing an issue removes the mark and cancels a re-check that has not finished yet.
3. Get a fresh capture of the page
Nothing to do if the page is tested again anyway: the next capture of the page in the issue's language is used automatically, whoever makes it.
If nobody tests the page, SurveyTester sends a reminder after two hours. For issues that came from a finding or a text change, you can then start a SurveyBot replay:
- in the issue: Re-check with SurveyBot in the Automatic re-check panel, or
- for all waiting issues at once: What's next → Run SurveyBot re-check now (see below).
The bot repeats the interview that reported the issue, with the same answers, test case, language and market, and stops as soon as it reaches the issue's page. Several issues reported on the same page share one bot run.
Manually reported issues cannot be judged by a bot. For them, the panel offers Repeat test run to this page instead: it opens the test run that reported the issue and repeats it up to the issue's page, so you can look at the page yourself. Your capture of the page is used for the re-check like any other.
4. Read the result and decide
When the result arrives, the issue history shows an Automatic re-check entry, and everybody who follows the issue is notified.
To verify a result, click the verdict (for example Looks fixed) in the Re-check column. A comparison opens with the verdict, its reason and two screenshots side by side: When reported (the page as the issue was found) and Re-check capture (the page the re-check judged). Click a screenshot to see it in full size. You can answer right there. The same two screenshots are also shown as small previews in the Automatic re-check panel of the issue.
Your answer:
- Confirm fixed sets the closing status of your project type. If your project type asks for a reason for closing, you are asked for it as usual.
- Still broken sets the issue back to an open status (the default value of the status field), so the programmers see it again. Without a re-check status in the project type, it removes the re-check mark.
Your answer is stored with the re-check.
Answering re-checks while testing
Testers can answer the re-check directly in the browser extension, while they see the page. In the Issues window, every issue of the page in the re-check status carries the question right below its description, in the overview and in the issue details. If the automatic re-check of a new capture already ran, its verdict is shown above the question, for example "SurveyTester: looks fixed". Then: Is this solved?
- Yes closes the issue with the closing status of your project type.
- No sets it back to an open status, so the programmers see it again.
If your project type asks for a reason for closing or re-opening, the extension asks for it before saving. Your answer is stored with the re-check, like an answer in the web interface, and the question disappears from the issue.
Findings from Auto-Detect follow the issue: the marker of a finding whose issue is under re-check is drawn in blue with "Fixed? Please re-check", and the marker of a finding whose issue is closed is no longer shown.
The question is shown to users whose role has the permission Can Answer Re-checks During Testing (Administration → Roles, area Issues). This permission is enough to answer "Yes", even without the permission to close issues. It applies to manual tests only, never to test links.
The "What's next" menu
The What's next button at the top of the issue list, left of the Auto-Detect findings, collects what the re-check left for you to do. It is shown to users whose role has the permission Can Use "What's next" (Administration → Roles, area Issues). The actions in the menu keep their own permissions: starting SurveyBot runs needs the right to test surveys, changing statuses needs the right to edit issues. The red number shows how many issues are waiting for you. For example:
- 17 fixed issues wait for a re-check - nobody has tested their pages again yet. → Run SurveyBot re-check now starts the replays after a short confirmation.
- 3 manually reported issues wait for a re-check by a person: followed by the issue numbers and pages. Click an issue to open it, then use Repeat test run to this page.
- 2 re-checked issues are still not fixed. → Change to "Not fixed" sets the open status of your project type on all of them at once.
Find re-checked issues with the result filter
The result filter knows four re-check criteria:
- Needs re-check (yes / no)
- Re-check: state - waiting for a capture, SurveyBot running, being checked, done, cancelled
- Re-check: verdict - looks fixed, still present, check manually
- Re-check: answer - confirmed fixed, still broken, or empty
The combination Re-check: verdict is not empty AND Re-check: answer is empty lists every result that still waits for your decision. Needs re-check is also available as a quick-add button.
AI suggestion for manually reported issues
Issues that a tester described in their own words have no rule SurveyTester could run again. For these, the re-check waits for the screenshot of the new capture and shows it to an AI together with the original screenshot and the issue description. The panel then shows an AI suggestion (looks fixed, still visible or cannot tell from the screenshot) with a short reason.
The suggestion is only a hint: the result stays please check manually. Administrators can choose the AI engine for this feature, or turn it off, under the AI feature Issue Re-check (see AI administration). The usage is metered like every other AI feature.
Good to know
- A result never changes the issue status. Only Confirm fixed, Still broken, the answer to Is this solved? in the browser extension or your own status change do.
- Same language. A re-check of the German version is only answered by a German capture.
- One capture serves all issues of the page. If ten issues sit on the same page, one test of that page re-checks all of them.
- Issues without a page cannot be re-checked automatically, because there is no page to capture.
- The SurveyBot replay follows the original route. If the route has changed since the issue was reported and the bot never reaches the page, the result is please check manually with the note that the page was not reached.
- Project-wide findings need new runs. Rotation, MaxDiff and item exposure re-checks only count test runs started after the re-check request. They stay waiting until enough new runs exist. A SurveyBot replay is not offered for them; start a few normal test runs or bot runs instead.
- A check that is switched off cannot re-check. If an Auto-Detect check is disabled for the project, its re-checks stay waiting. See Auto-Detect issues.
- The accepted finding in the Auto-Detect findings list keeps its accepted mark; the re-check result is shown with the issue. On the survey page in the browser extension, its marker turns blue (Fixed? Please re-check) while the issue waits for a re-check, and disappears once the issue is closed.