Respondent experience management Part 2

Note for the readers: This article is published in two parts. This is the second part. If you did not get a chance to read the first one, I encourage you to read the first piece before continuing to read this article as they are complementary. 

In the first piece, we have discussed the importance of respondent experience and the first two stages of the ERMBI model. Now, we will explain the rest of the stages. 

3. Pre-live experience

Before discussing the pre-live experience, it is important to highlight that sometimes the first two stages cannot be implemented for various reasons. For example, a lack of budget will prevent the first stage “explore the respondent” and ultimately the second stage would have the same status. It is not possible to propose a “relevant design” without “exploring the respondent”. In this case, we can still move to complete a short version of the model which is the “Partial ERMBI model” Measure, Build and Implement.

Now comes the pre-live experience. The idea of this process is to monitor a project basis to get real-time feedback. Before starting the live fieldwork and during the pilot phase is a good opportunity to measure the experience. You can ask a few questions at the end of the pilot interview, the suggested questions are called the Respondent Experience Metrics (RXM):

It gives a high-level assessment of the survey design. Usually, the pilot has a small sample, but looking at the respondent experience metrics qualitatively helps to improve the questionnaire design and the experience on a project basis. The output will suggest the improvement actions.

4. Post-live experience

After exploring the respondent, creating a relevant design and measuring the pre-live experience, it is a time to assess your effort. Establish a tracking program to get the respondent feedback after every live interview, this type of assessment should be very short and includes the “RXM” (illustrated above in table 1) which will allow you to have a “respondent experience index" for each project. 

Apply the following formula to calculate the Respondent Experience Index (RXI):

 RXI=(E+C+L)-B

RXI Range (-100 : +100)

Where;

E: effort (% of “easy” + “JR” responses)

C: clarity (% of “clear” response)

L: length (% of “JR”+ “Shorter than expected” responses)

B: burden (% of “yes” response) 

5. Building a espondent experience (RX) database

While the tracking program (post-live experience) is running, create an internal database where you can store the respondent experience metrics along with the respondent experience index for each project (the post-live experience will feed the respondent experience database). Additionally, make sure you add the profile fields in your database, so the fields would be:

• Project name

• Client name

• Industry

• Geographical coverage (city/ region/ country)

• Fieldwork date(start/end)

• Fieldwork duration

• Target respondent

• The planned length of interview (Est minutes)

• The actual length of interview (Avg)

• The actual length of the interview (Min)

• Actual length of interview (Max)

• The incentive given (Y/N)

• Incentive type

• Data collection method (CAPI/CATI/CLT etc)

• Effort score (% of “easy” + “JR” responses)

• Clarity score (% of “clear” response)

• Length score (% of “JR”+ “Shorter than expected” responses)

• Burden score (% of “yes” response)

• RXI 

It is important to use any database application that can help you to set search criteria and retrieve the data easily.

6. Implement

Now the processes start to reach an end, after exploring the respondent, measuring the real experience and building the respondent experience database; we need to bring the respondent experience in the research to life, applying the following actions supports this stage:  

Firstly, before proposing any research design refer to the RX database to gain understanding. Search for any previous project which seems similar to the one in the proposing stage, compare the respondent experience metric and index with the implemented design (i.e. method, LOI) and then apply the learning to the new project. As you can see the model will work as an optimising tool for future research design.

Another benefit of applying the model is that after a period of time (e.g. year) you would have “a respondent experience index” on different levels (e.g. industry, data collection method, LOI). It could be an indicator of how to engage the respondents and get high-quality responses. It could be a unique selling point or differentiator across research providers.

Secondly, utilise technology. Explore and utilise technology that makes the interview easier and effortless. For instance, survey platforms could introduce a pre-loading feature [f] that reduces the length of the interview and respondent effort improving the overall experience

Beacon [g] is another technological tool, the applications of this technology in market research can vary. It could be a reminder to the respondent to participate in any research that focuses on a real-time experience in a specific location so that a tool can run an NPS transaction survey. This helps the researcher to get accurate responses by reducing the respondent’s effort in memorising the things around the experience. Moreover, it creates a good feeling and enhances the respondent experience. 

Third, questionnaire gamification [h] as stated by Gamification Research Network -GRN [e], in the academic realm, several studies in various contexts have shown that gamification can be an effective approach to increase motivation and engage users or participants in a given activity. 

In conclusion

The respondent experience has to be at the top of researchers’ priorities. Memorable or at least reasonable experience is a vital requirement to get quality of responses, deliver reliable insights and recommendations. ERMBI model is a trial to manage and achieve such an experience. 

References

[e] Gamification Research Network, http://gamification-research.org/2016/09/how-to-gamify/                See e.g. Hamari et al. 2014; Morschheuser et al. 2016 for reviews).

Morschheuser, B., Hamari, J., & Koivisto, J. (2016).  Gamification in crowdsourcing: A review.In proceedings of the 49th Annual Hawaii International Conference on System Sciences (HICSS), Hawaii, USA, January 5-8, 2016.

Hamari, J., Koivisto, J., and Sarsa, H. (2014). Does Gamification Work? – A Literature Review of Empirical Studies on gamification. In proceedings of the 47th Hawaii International Conference on System Sciences, Hawaii, USA, January 6-9, 2014.

Notes:

[f] Pre-loading feature allows users to save known information for the target respondents before starting the live interviews (e.g. demographic variables) and then attach the same information with the respective respondent responses after the completion.

[g] Beacon is a tool to pinpoint the location of a portable device (smartphone) holder and then triggers (push messages or remainders) can be sent based on predefined criteria for the location.

[h] Gamification is a technique of using the games out of the context of playing. The core concept of gamification in marketing research is all about making the questions more “game-like” which can take several forms.