SurveyBots are fast. Start a test across 5 languages, 2 markets and 3 test cases, and SurveyTester creates 30 jobs at once. Without limits, all their bots hit the survey at the same moment. Some survey systems cannot handle that: they slow down, block the traffic, or hand out the same interview ID to several bots that arrive together.
Without SurveyTester you would have to stagger such a test by hand: start a few interviews, wait, start the next few, and keep track of every language and market yourself. In SurveyTester, two settings in the Test Survey dialog do this for you, and both apply to the whole project, not to a single job.
The two settings
| Setting | What it does | Default |
|---|---|---|
| Max Concurrent SurveyBots for the Project | The number of SurveyBots that may run at the same time, across all jobs of the project. | 10 |
| Delay between SurveyBot starts (sec.) | The minimum time between two SurveyBot starts in the project. Use 0 for no delay, 60 is the maximum. | 1 |
Both settings work across all languages, markets and test cases. If you start 10 jobs with a limit of 10, you get 10 bots at the same time, not 100. If you start a second test on the same project while the first one is running, its bots count toward the limit too.
How to set the limits
Step 1: Open the Test Survey dialog
Open your project, open Test Survey and choose SurveyBot Testing. Select the languages, markets and test cases as usual.
Step 2: Set the values
Go to the Test Runs box. Enter the number of test runs, then adjust Max Concurrent SurveyBots for the Project and Delay between SurveyBot starts (sec.). The number next to the first field shows the highest value your company allows.
Step 3: Start the test
Start the SurveyBots. SurveyTester now starts one bot at a time, with at least the delay between them, until the limit is reached. Each time a bot finishes, the next one starts.
The bots do not work through one job after the other. Each free bot takes a random job, so languages, markets and test cases are mixed from the start and you see results for all of them early on.
Which values should I use?
| Situation | Recommendation |
|---|---|
| Several bots get the same interview ID | Keep the delay at 1 second. If it still happens, try 2 to 5 seconds. |
| The survey system gets slow or shows errors under load | Lower the limit. |
| A small test on a robust survey system | Keep the defaults. |
| Many combinations and the test takes too long | Raise the limit, if the survey system can handle it. |
Note
Both settings make large tests take longer. With a delay of 1 second, 300 test runs need at least 5 minutes just to start. The real gap between two starts can be 1 to 2 seconds longer than the delay when no other work is waiting.
Note
The limit only covers tests started from the Test Survey dialog. SurveyBot runs that SurveyTester starts by itself, for example for sample-only projects or issue re-checks, have their own settings and are not counted.
For administrators: the company limit
The highest value users can enter for concurrent SurveyBots is set in Company Settings under Maximum Concurrent SurveyBots per Project. Use it to set a sensible upper limit for your whole team.
