Comparing a project with a previous wave

Most surveys are not written from scratch. A tracking study comes back every quarter, a questionnaire is translated for a second market, or a client asks for "the same survey as last year with three new questions". The new script is a copy of an old one, and the interesting question for the tester is: what changed? The Compare Projects page links the new project to the one it was built from and lists every difference between the two surveys, so that the planned changes are easy to confirm and the unplanned ones stand out.

Why comparing by hand is hard

Two scripts that look the same are rarely identical. Between two waves a question is reworded, an answer option is dropped, a routing condition is tightened, a page is split in two. None of that is visible in the survey link, and reading two questionnaires side by side for an hour is exactly the kind of work in which the one changed word slips through.

With a linked compare project SurveyTester does the comparison for you:

• The Compare Projects page summarises what differs: new and missing pages, questions that were added, removed, moved or renamed, changed question texts, changed answer options and grid rows, and routing that differs.
• Pages & Routing shows the pages of the paired project next to the pages of the current one, and the routing diagram opens as two diagrams side by side, aligned like a text comparison.
• Test runs can be repeated in the paired project: a run recorded in the old wave is replayed against the new script with the same answers.

Where the setting is

Open the new project and go to Project Details › Compare Projects. The page is only shown to users whose role has the permission Can compare projects.

Step by step

1. In Project Details › Compare Projects, click the field Compare with project. A project picker opens with the same tree as the main project menu: your companies, their folders and the projects inside.
2. Type a few letters into the search box to narrow the tree, or open the folders by hand. The project you are currently working in is not offered - a project cannot be compared with itself.
3. Looking for an older wave? Turn on Show closed projects. Closed projects are the usual comparison targets, because the previous wave has normally been signed off and closed.
4. Click the project. The picker closes, the field shows Company › Folder › Project name, the link is saved immediately, and the comparison starts.
5. Read the summary line: it says what kind of differences were found, or that the two projects are the same. Below it, one section per kind lists the differences, each with the page and question it belongs to and the detail - for a changed text the old and the new wording in one line, with what was removed struck through and what was added underlined.
6. Go through the list. Switch a difference to Planned when it was intended; it moves out of the Open view. For an unintended difference click Create issue: an issue opens, prefilled with the page, the change and both texts, and stays linked to the difference. The row then shows the issue as a chip with its number (#12); a click opens the issue, a closed issue is shown greyed and struck through. A difference can have several issues - one per language, or a second one when the first was closed too early - so Create issue stays available next to the chips.
7. Not sure what a line means? Click Side by side: a dialog shows the page of this project on the left and the matching page of the compare project on the right, each as its latest screenshot, with the difference and its Planned switch and Create issue button above them. The arrows in the title bar walk through the list without closing the dialog.

To remove the link, click the small × next to the selected project.

What you get

The Compare Projects page is the overview: the filter chips Open, Planned and All switch between the differences that still need a look and those already confirmed, and a quiet line tells you when planned changes have been made and no longer differ. Show in Pages & Routing jumps to the paired view: next to each page of the current project you see the matching page of the paired project, and the routing diagram opens as two diagrams side by side, pages that exist on one side only marked with a dashed outline. In Test Runs, the dialog Repeat test run offers to repeat a run in the paired project.

Good to know

• One compare project per project. The link points in one direction: set it on the new wave and let it point at the old one.
• One language at a time. A multilingual project is compared in one of its languages; a language selector at the top of the page switches between them, and the page remembers your choice. Pages, questions, answer codes and routing are the same in every language, so those differences and their planned marks are shared. Changed texts belong to the language you are looking at, and an issue created from the page carries that language. When the compare project has no version in the chosen language, its default language is used and a note says so. A text is only compared where both projects have one in that language: a question the previous wave never asked in Spanish shows no Spanish text differences.
• Markets, optionally. When the project has markets, a second selector next to the language picks one. The comparison then uses the market's own texts where it has them and the language's texts everywhere else, and the compare project is read through its market of the same name. All markets compares the language's texts. An issue created from a market view carries the market, and planned marks on texts are kept per market.
• The counts are switches. Each count in the summary line - page changes, question changes, text changes, answer option changes, routing changes - shows or hides its section below. Switch the ones you do not need right now off; the page remembers them, like the Open / Planned / All choice, so you find the same view when you come back.
• Search. The search box next to the filter chips narrows the list to the differences that contain every word you type, in page and question names, answer codes and texts. Escape or the × clears it.
• Piped texts are compared as templates. A question that pipes a brand or a stimulus into its text ("Rate your Audi", "Rate your Tesla") shows a different text in every run. SurveyTester notices that from the text history of the project and compares the template instead - "Rate your {pipe-in}" - so two waves with the same script are not listed as a text change. The placeholder is shown as a small chip; its tooltip lists the values seen. A text that varies as a whole, with no common part, is not compared at all.
• Rotated questions are not compared by order. When the answer options or the grid rows of a question rotate - detected from the test runs or set as an expectation - a different order between the two waves is not a difference and is not listed. Added, removed and changed options are still reported.
• Routing is observed from test runs. A route that one side has not shown yet is listed as not seen yet, together with the number of test runs that would settle the question, and it is not counted as a change. Test the new wave a little more and the entry resolves itself.
• Pick the test runs by milestone. Large projects go through phases - a pilot, the first field wave, a relaunch - and you usually want to compare one phase, not everything the project ever ran. Under the project picker, each project has a From milestone and a To milestone selector listing its milestones. Only test runs started between the two milestones count for the routing comparison and for the page screenshots of that project; leave a selector empty for no bound. The range is saved with the comparison, so it is the same for everyone who opens the page, and the summary line shows the dates in use. Texts, pages and answer options are always compared as they are now - the range only decides which runs the routing is read from.
• Decisions survive. Planned marks are kept per difference across recomputations and project copies. When the text changes again, that is a new, open difference.
• Issues from this page can be re-checked automatically. An issue created here remembers which difference it was made for: the page, the question, the answer option and the text in the compare project. When the issue is later set to Needs re-check (see Automatic re-check of fixed issues), the next test run of that page is judged against the compare project: a changed text counts as fixed when the compare project's wording is back, a question or answer option that was new counts as fixed when it is gone, one that was missing when it is back. Differences in order, routing or question type still need a look by a person. Deleting an issue removes it from the row.
• Closed projects stay selectable. You can compare with a closed project at any time; the closed project itself is not changed by it.
• A project you can no longer see - deleted, or moved to a company you have no access to - is shown by its id in the field. Remove it with the × and pick another project.
• The best moment to set the link is right after creating the new wave, before the first test runs; see the project workflow in Project best practices.