CatalystMR strives to provide the best possible research experience for our clients. This commitment to quality is why the industry’s top research companies rely on CatalystMR for their CATI and online survey programming, hosting, reporting and study management needs.
CatalystMR ensures our clients’ high standards of quality are met by applying the following quality control procedures.
Step 1: Planning and scrutinizing before survey programming
Instrument and instruction validation
Once you provide our team a survey instrument – online survey, a paper questionnaire, or CATI survey, our team will scrutinize your document for logical inconsistencies, incomplete programming instructions, and overall survey flow. Your assigned project manager then reviews all other project related information such as special programming requirements, sample size, incidence, quotas, final deliverables, etc.
Resource allocation and project planning
Once the full scope of the project has been realized, a time-line is developed outlining the most efficient and cohesive allocation of resources. Project details are then recorded in a central database for team reference, resource cooperation, and a consistent flow of project information between our nationwide team.
Respondent completion monitoring
CatalystMR’s data collection tools allow us to collect an array of information about the respondent, including IP address, browser version, start and end date/time and the length of interview.
Digital Fingerprinting (natively integrated): Digital fingerprinting technology improves data quality by accurately identifying and flagging suspect respondents as defined by you in real-time from any sample source across the market research industry.
With your input, CatalystMR establishes a minimum and average survey completion time for your study and assigns a time-score for each survey respondent. This practice is a great way to manage your data so that your analysis and findings are based on respondents who actually participated in your study.
Access to software architects
CatalystMR benefit from closes relationships with the software developers and technical support staff that support our operations. Our close relationships bring experienced market research software specialists to your projects at every turn.
Step 2: Survey programming and testing
Constant testing and client involvement
Early in the survey programming process, CatalystMR provides working examples of complicated and/or crucial sections of the survey for client approval. This allows for feedback and customization from the client before proceeding on to final programming.
When the initial phase of survey programming is complete, staff members test and evaluate the survey for the following:
- skip logic
- single response vs. multiple responses
- exclusive responses in multiple response lists
- exception codes
- consistency in the onscreen presentation of the online survey
- spelling / grammatical errors
- text substitution / question back-referencing
- question and response rotations
- survey terminate points
- survey selection processes
- proper survey suspending and resuming
- survey quotas
- survey error messages
- interviewing instructions and respondent instructions
Data verification, before collecting data
Prior to an online or mobile web survey fielding, CatalystMR runs a Random Data Generator (RDG) that creates a sample data file against which marginals are generated and checked. Survey quota tallies, skip patterns, and checks / limits are then verified by reviewing the data generated by the RDG process.
Step 3: working the plan and project support
Invitation creation and list validation
With testing completed, CatalystMR creates passwords for online and mobile web surveys through a unique password generator. Once a list of passwords has been created, project managers confirm unique IDs through a strict de-duplication process.
Next, the project manager loads e-mail addresses, passwords, and the remaining contents of the email survey invitation letter into CatalystMR email management tool. In addition to checking for structurally valid e-mail addresses, our system provides for the scheduling of invitations over hours, days, or weeks.
11-pronged Approach to Online Survey Respondent Quality Control
CatalystMR is consistently pushing the envelope when it comes to panel data quality. While we feel that the online panel sample industry has done a remarkable job in the last few years improving its transparency and introducing technology breakthroughs such as digital fingerprinting, we don’t rely on others to ensure that your survey data doesn’t contain speeders, duplicates, and computer bots. While we work with only best online sample suppliers who go through our detailed review process including submitting their response to the ESOMAR 26, we have our own 11-pronged approach to improving online panel sample data quality. We’ve been there..did it, done it. Don’t settle for bad data… Don’t pay for bad data!
CatalystMR data collection tools allow us to collect an array of information about the respondent, including IP address, browser version, start and end date/time and the length of interview.
As part of our 11-Pronged Approach to Online Survey Respondent Quality Control, CatalystMR (with your approval) establishes a minimum survey completion time for your study and assigns a time-score for each survey respondent. For those respondents who don’t meet the minimum threshold online web survey time to completion, they don’t count towards your quota. This means you never see speeders in your survey data nor do you pay for them because they are caught before they are recorded. In the end, your analysis and findings are based on respondents who actually participate in your study, not those who are attempting to game the system.
Real-time online sample quality control:
Cheaters or Satiators don’t show up in your data thus they don’t show up in your quota or real-time tabulation reports. You’ll know who they are only through your real-time disposition reports.
- Server side password protection: One password per respondent contacted. Once that respondent completes, they’re survey status is flagged as complete and thus cannot reenter the survey.
- Digital Fingerprinting: Digital fingerprinting technology improves data quality by accurately identifying and flagging respondents as defined by you in real-time from any sample source across the market research industry.
- Respondent (client) side password protection. What happens if two panel suppliers invite the same respondent and each provides their own unique password? CatalystMR set’s the original password in the respondent cookie directory so that if the respondent comes back to the survey with a new unique password, we check their cookie directory to see if they’ve already responded to that survey. If they have, we submit the original password to the server and thus, the web survey respondent receives a message letting them know that they’ve already completed the online survey.
- CAPTCHA Code to tell a computer from a human. What happens when a computer bot takes a web survey… Don’t let yourself fall prey to schemes to reap incentives on the masses. We have a password that we display to the respondent in the form of an image. Because the password is an image and not text, the bot cannot read the code and thus doesn’t know how to get past the question.
- Speed traps: Cheaters don’t cheat slowly and thus don’t reach the allowable threshold of time to take the web survey. If a respondent takes an online survey too quickly they don’t count toward quota and thus don’t show up in your data.
- Trap Questions: If respondents don’t provide consistent answers or provide answers that don’t make sense they don’t end up in your online survey data.
Post processing panelist quality control:
Who wants to pay for online panel respondents who type “alsjdjd” into their open-ended response fields? Who wants duplicate survey respondents in their survey data?
- Survey data cleaning by comparing multiple variables associated with IP address
- Close inspection of open-end unstructured open-end data
- Close analysis of straightliners
Non-stop project support and monitoring
Once a online survey goes live, we provide comprehensive project support and monitoring. Your project manager will always be available while the study is in-field.
Using automated scripts, CatalystMR project managers monitor, in real-time, data being collected by the survey to ensure data integrity and consistency between data, quota, and the sample files.
Online survey system quality control and error checking
- Real-time error checking: Full screen control of error message text, required responses per question, per screen, and types of acceptable variables on screen (text, numeric, and multiple response questions).
- Requiring responses: Specific questions or the entire online web survey can be customized to require or not require an answer(s).
- Error checking is customizable by question and by screen: Error messages can be turned on or off depending on the web survey.
- Displaying error messages: Error checking will display an error message specific to each question and/or attribute on a given screen.
Project managers will field e-mail and phone inquiries from respondents. We believe that active and timely support results in positive survey experiences that preserve the quality of the total sample pool and the satisfaction of your respondents.
Report and data tabulation quality assurance
All tables and reports undergo a strict quality control testing and verification procedures. Using utilities developed specifically for CatalystMR data collection and reporting software we validate the survey data that was collected, entered, and reported before the results are presented to you or your client.
So what are you waiting for? Try CatalystMR’s hassle free online bid form: click here