About_CatalystMR

Quality Control

CatalystMR strives to provide the best possible research experience for our clients. This commitment to quality is why the industry’s top research companies rely on CatalystMR for their CATI and online survey programming, hosting, reporting and study management needs.

CatalystMR ensures our clients’ high standards of quality are met by applying the following quality control procedures.

Step 1: planning and scrutinizing before survey programming

instrument and instruction validation

Once you provide our team a survey instrument - online survey, a paper questionnaire, or CATI survey, our team will scrutinize your document for logical inconsistencies, incomplete programming instructions, and overall survey flow. Your assigned project manager then reviews all other project related information such as special programming requirements, sample size, incidence, quotas, final deliverables, etc.

resource allocation and project planning

Once the full scope of the project has been realized, a time-line is developed outlining the most efficient and cohesive allocation of resources. Project details are then recorded in a central database for team reference, resource cooperation, and a consistent flow of project information between our nationwide team.

respondent completion monitoring

CatalystMR’s data collection tools allow us to collect an array of information about the respondent, including IP address, browser version, start and end date/time and the length of interview.

Upon request, CatalystMR will establish a minimum and average survey completion time for your study and assign a time-score for each survey respondent. This practice is great way to manage your data set so that your analysis and findings are based on respondents who actually participated in your study.

access to software architects

Throughout survey programming and management, members of the CatalystMR benefit from a close relationship with the software programming and technical support staffs. This relationship brings experienced market research software specialists to your projects and promotes the general development of the software.

Step 2: survey programming and testing

constant testing and client involvement

Early in the survey programming process, CatalystMR provides working examples of complicated and/or crucial sections of the survey for client approval. This allows for feedback and customization from the client before proceeding on to final programming.

When the initial phase of survey programming is complete, staff members test and evaluate the survey for the following:

  • skip logic
  • single response vs. multiple responses
  • exclusive responses in multiple response lists
  • exception codes
  • consistency in the onscreen presentation of the online survey
  • spelling / grammatical errors
  • text substitution / question back-referencing
  • question and response rotations
  • survey terminate points
  • survey selection processes
  • proper survey suspending and resuming
  • survey quotas
  • survey error messages
  • interviewing instructions and respondent instructions

data verification, before collecting data

Prior to an online web survey fielding, CatalystMR runs a Random Data Generator (RDG) that creates a sample data file against which marginals are generated and checked. Survey quota tallies, skip patterns, and checks / limits are then verified by reviewing the data generated by the RDG process.

Step 3: working the plan and project support

invitation creation and list validation

(Web surveys only)

With testing completed, CatalystMR creates passwords for online web surveys through a unique password generator. Once a list of passwords has been created, project managers confirm unique IDs through a strict de-duplication script.

Next, the project manager loads e-mail addresses, passwords, and the remaining contents of the email survey invitation letter into CatalystMR email management tool. In addition to checking for structurally valid e-mail addresses, our system provides for the scheduling of invitations over hours, days, or weeks.

8-pronged Approach to

Online Survey Respondent Quality Control

CatalystMR is consistently pushing the envelope when it comes to panel data quality. While we feel that the online panel sample industry has done a remarkable job in the last few years improving its transparency and introducing technology breakthroughs such as digital fingerprinting, we don’t rely on others to ensure that your survey data doesn’t contain speeders, duplicates, and computer bots. While we work with only best online sample suppliers who go through our detailed review process including submitting their response to the ESOMAR 26, we have our own 8-pronged approach to improving online panel sample data quality. We’ve been there..did it, done it. Don’t settle for bad data…Don’t pay for bad data!

CatalystMR data collection tools allow us to collect an array of information about the respondent, including IP address, browser version, start and end date/time and the length of interview.

As part of our 8-Pronged Approach to Online Survey Respondent Quality Control, CatalystMR (with your approval) establishes a minimum survey completion time for your study and  assigns a time-score for each survey respondent.  For those respondents who don’t meet the minimum threshold online web survey time to completion, they don’t count towards your quota. This means you never see speeders in your survey data nor do you pay for them because they are caught before they are recorded. In the end, your analysis and findings are based on respondents who actually participate in your study, not those who are attempting to game the system.

We Employ:

Real-time Online Sample Quality Control:

Cheaters or Satiators don’t show up in your data thus they don’t show up in your quota or real-time tabulation reports. You’ll know who they are only through your real-time disposition reports.

  • Server side password protection: One password per respondent contacted. Once that respondent completes, they’re survey status is flagged as complete and thus cannot reenter the survey.
  • Respondent (client) side password protection. What happens if two panel suppliers invite the same respondent and each provides their own unique password? CatalystMR set’s the original password in the respondent cookie directory so that if the respondent comes back to the survey with a new unique password, we check their cookie directory to see if they’ve already responded to that survey. If they have, we submit the original password to the server and thus, the web survey respondent receives a message letting them know that they’ve already completed the online survey.
  • CAPTCHA Code to tell a computer from a human. What happens when a computer bot takes a web survey….Don’t let yourself fall prey to schemes to reap incentives on the masses. We have a password that we display to the respondent in the form of an image. Because is an image and not text, the bot cannot read the code and thus doesn’t know how to get past the question.
  • Speed traps: Cheaters don’t cheat slowly and thus don’t reach the allowable threshold of time to take the web survey. If a respondent takes a online survey too quickly they don’t count toward quota and thus don’t show up in your data.
  • Trap Questions: If respondents don’t provide consistent answers or provide answers that don’t make sense they don’t end up in your online survey data.

Post Processing Panelist Quality Control:

Who wants to pay for online panel respondents who type “alsjdjd” into their open-ended response fields? Who wants duplicate survey respondents in their survey data?

  • Survey data cleaning by comparing multiple variables associated with IP address
  • Close inspection of open-end unstructured open-end data
  • Close analysis of straightliners

non-stop project support and monitoring

Once a online survey goes live, we provide comprehensive project support and monitoring. Your project manager will always be available while the study is in-field.

Using automated scripts, CatalystMR project managers monitor, in real-time, data being collected by the survey to ensure data integrity and consistency between data, quota, and the sample files.

Online survey system quality control and error checking

  • Real-time error checking: Full screen control of error message text, required responses per question, per screen, and types of acceptable variables on screen (text, numeric, and multiple response questions).
  • Requiring responses: Specific questions or the entire online web survey can be customized to require or not require an answer(s).
  • Error checking is customizable by question and by screen: Error messages can be turned on or off depending on the web survey.
  • Displaying error messages: Error checking will display an error message specific to each question and/or attribute on a given screen.

respondent support

For online surveys, your project manager will field e-mail inquiries from respondents. We believe that active and timely support results in positive web survey experiences that preserve the quality of the total sample pool and the satisfaction of your respondents.

report and data tabulation quality assurance

All tables and reports undergo a strict quality control testing and verification procedures. Using utilities developed specifically for CatalystMR data collection and reporting software we validate the survey data that was collected, entered, and reported before the results are presented to you or your client.

So what are you waiting for? Try CatalystMR’s hassle free online bid form: click here