Using the SurveyToGo Dummy Data Generator

Guide to the Dummy Data Generator—programmatically generate hundreds of test interviews to validate script logic and data structure before fieldwork.

Overview

When creating complex surveys that have many branching/skipping/filtering instructions in them, most customers run a test phase where test interviews are filled out so that the data processing department can run checks on the test data before going into field.

A great way to enhance this “test phase” is by utilizing the “SurveyToGo: Dummy Data Generator” that enables you to programmatically generate tens or hundreds of test interviews and then export them into your preferred data format.

The SurveyToGo: Dummy Data Generator allows you to:

• Set the amount of test-interviews to generate
• Run the test with or without validation rules
• Define the Start and End questions for the Dummy Data Generator to generate data for
• Fine tune the answers and weights of random answers if necessary
• Export the test data to all supported formats
• Does not “eat up” any of your interview balance

To use the SurveyToGo: Dummy Data Generator:

1. Run the Dummy Data Generator
2. Using the SurveyToGo: Dummy Data Generator
   1. Define the Start and End questions for which the Dummy Data Generator will generate data.
   2. Fine tune the answers and weights if necessary
3. Use best practices for generating data for complex CAPI surveys
4. Save & export the test data using the SurveyToGo: Studio

Online Videos

Direct Youtube link (for full screen): https://youtu.be/jPshQeATLbw

Direct Youtube link (for full screen): https://youtu.be/vuwDZFzo754 

Step 1: Run the SurveyToGo: Dummy Data Generator

To run the SurveyToGo: Dummy Data Generator:
 

1. Open the survey you wish to generate data for:
   
2. In the Preview section - Click on Run in Emulator 
3. Once the emulator comes up, click the “Dummy Data Generator” menu option:
4. This will bring up the SurveyToGo: Dummy Data Generator main screen:
5. That’s it!

Step 2: Using the SurveyToGo: Dummy Data Generator

After starting the SurveyToGo: Dummy Data Generator you will be see the following screen:

The screen contains the following options: 

After deciding on the number of interviews and whether to use validation rules or not, you can simply click the “Generate” button to generate the dummy data:

During the test run and after the run is complete you will see the Stats-Tab filling up with the various statistics. Once the run is complete you will see a notification prompt and the “View Data” and “Save Data” buttons will become active. The quota summary tab will also list the various quotas that were filled:

You can click the “View Data” button to bring up the various data fields that were generated during the data generation process:

there are 2 important fields other than the actual data that should be considered:

1. UsrUnq: Will hold the outcome status for the interview
2. StopQ: If the interview was either filtered, canceled or dropped this field will contain the question that caused the termination of the interview.

You can also click the “Save Data” button to save the data to an “.sbj” file:

Saving the data to an “.sbj” file will allow you to later export the generated data through the standard SurveyToGo: Studio export wizard as shown in Step 6 of this how-to.

Step 3:  Define the Start and End questions for Dummy Data Generator to generate data for

The Dummy Data Generator  enables you to specify at which question to start the generation and at which question to stop. This is useful as many times scriptwriters will use the Dummy Data Generator to only check parts of the survey script, something that was difficult to do in previous versions.

To allow this just add the following callback function to the “Advanced scripts” section of your survey which can be accessed by clicking on Advanced scripts from the “Scripts” tab of any question in the survey.

This callback will be called when the Dummy Data Generator initializes and allows you to set the ioParams.StartQuestionIndex and ioParams.EndQuestionIndex to whatever question index you need:

Note: This is a callback function that is referenced directly by the Dummy Data Generator. If the engine finds the function in the Advanced Scripts it would use what is defined in it rather than the default behavior. Do not call this function directly in any of the questions' scripts.

Step 4: Fine tune the answers and weights if necessary

Complex CAPI surveys are usually composed of a screening part and a main questionnaire. The screening part of a survey can sometimes be so complex that by using randomly generated answers it would be nearly impossible to complete even a single interview. For these situations you have the ability to intervene and manipulate the random data generation process in the following ways:

1. Ability to set the specific answer for specific question instead of the randomly generated data used by the SurveyToGo: Dummy Data Generator. For example, for SurveyToGo: Dummy Data Generator, always choose “Yes” in the question that asks if you agree to participate in the interview.
2. Ability to alter the weights used when randomly selecting an answer. For example, for a question with 10 answers, give a 40% chance that answer 2 is chosen (instead of the default even 10%). This will allow you to increase the chance of completing interviews while still checking the scenarios of other answers.

To fine tune these 2 aspects, you can write the following 2 call back functions in the “Advanced scripts”:

OnGetDummyGeneratorValue – this call back allows you to alter the answers. It gets called for each question that the generator is generating data for. If you decide to manipulate the answer, you should return true. Otherwise, if you wish for the generator to randomly select, return false.

OnSetDummyGeneratorWeights – this call back allows you to alter the weights of answers for the random selection of answers. It gets called for each question that the generator is generating data for. If you decide to manipulate the weights of answers, you should return true. Otherwise, if you wish for the generator to use the default weights, return false. The default weight given to each answer is 1. The array of answers is a zero-based array meaning that the weight of answer 1 is in ioArray[0] and the weight of answer 3 is in ioArray[2]. You do not have to set the weights for all answers, only to the ones you wish to alter.

Note: The above functions are callback functions that are referenced directly by the Dummy Data Generator. If the engine finds the function in the Advanced Scripts it would use what is defined in it rather than the default behavior. Do not call the functions directly in any of the questions' scripts.

Step 5: Best practices for generating data for complex CAPI surveys

It is recommended that you follow the below best practice when generating dummy data for complex CAPI surveys that include many logical validation, branching, skipping and filtering code. The best-practice suggested below enables you to effectively generate enough completed interviews while leaving in place as many validation logic checks as possible. The main issue with complex CAPI surveys is that as the validation and survey logic is meant to check and cross-check answers, it is sometimes hard to get even a single interview completed by filling the interview with random answers as almost all interviews will be either filtered, canceled or dropped. To overcome this, the following best practice is suggested:

1. Run the SurveyToGo: Dummy Data Generator with only 10 interviews:
   
2. Notice the amount of submitted/canceled/filtered/dropped interviews. If you have above 3 submitted interviews, no need to alter anything. If less than 2 interviews are submitted notice whether most are dropped, canceled or filtered:
   
3. Then, open the “View Data” and search for the “Stop Q” column and notice which question shows up most and what the status column reads (canceled, filtered or dropped):
   
4. In the above case it is clear that Q2 is the one not allowing the SurveyToGo: Dummy Data Generator to continue and since the status reads “Dropped” we know it is a validation logic that is preventing the SurveyToGo: Dummy Data Generator to continue. You can then decide to either:
   1. Run the generation without validation logic checks:
      
   2. Or fine tune the generation of the answer of Q2 as shown in “Step 3″ of this how-to.
5. Once dealt with Q2, run the generation process again for 10 interviews and watch again for the ratio of submits vs canceled/filtered/dropped. If the ratio is still not good, run #3-4 again until you generate the desired submit/no-submit ratio.
6. Once the desired ratio is achieved you can then change the number of interviews to a higher number.

Step 6: Save & export the test data using the SurveyToGo: Studio

Saving the data to an “.sbj” file will allow you to later export the generated data through the standard SurveyToGo: Studio export wizard. To save the data to an “.sbj” file, click the “Save Data” button:

Then, to export a saved “.sbj” file to any of the SurveyToGo: supported formats, simply start the SurveyToGo: Studio export wizard for this survey (more on how to export here:  https://support.dooblo.net/hc/en-us/articles/208294565-How-To-Export-Survey-Results and on the options screen, specify the location of the “.sbj” file:

This will use the data in the “.sbj” file as the source of the data to export instead of the actual survey data. By using this method you can export the data collected with the SurveyToGo: Dummy Data Generator to any of the support SurveyToGo: export formats.