Testing (III): questionnaires
Questionnaires for user testing VR & AR Source
Go and look for a questionnaire that suits your goals. " Measuring Presence: A Guide to Current Measurement Approaches”, J. van Baren & W. IJsselstein (2004) Tip: go to the appendix. That will calm down your brains a bit.
A1: QUESTIONNAIRE ON PRESENCE AND REALISM (part 1)
Items 1-8 are rated on a 4-point scale: 0 (none) 1 (2-50%) 2 (50-75%) 3 (75-100%).
If one’s level of presence in the real world is 100%, rate your level of presence in this virtual world. (please encircle 0 1 2 3 )
How strong was your sense of presence? (please encircle 0 1 2 3 )
Did you feel you could reach into the virtual environment and grasp an object? (please encircle 0 1 2 3 )
How realistic did the virtual world appear? (please encircle 0 1 2 3 )
How realistic were depth and volume? (please encircle 0 1 2 3 )
How realistic were the virtual world’s reactions to your actions? (please encircle 0 1 2 3 )
When exploring the virtual space, did the objects appear too compressed or too magnified? (please encircle 0 1 2 3 )
Did the virtual objects appear geometrically correct, did they seem to have the right size and distance in relation to yourself and other objects? (please encircle 0 1 2 3 )
A2: QUESTIONNAIRE ON PRESENCE AND REALISM (part 2)
The two items below are rated on a 5-point scale: Very satisfactory (5) Moderately satisfactory (4) Neutral (3) Moderately unsatisfactory (2) Very unsatisfactory (1)
Overall, how would you rate the sense of presence generated by the environment? (please encircle 1 2 3 4 5 )
Overall, how would you rate the degree of realism achieved by the virtual environment? (please encircle 1 2 3 4 5 )
B1: NOWAK QUESTIONNAIRE on co-presence
From here on, all items below can be rated on a 5-point scale where you:
1= Strongly disagree with the statement 2= Disagree 3= Neutral, please leave a short remark why 4= Agree 5= Strongly agree.
Items assessing perceived other’s copresence:
My interaction partner was intensely involved in our interaction. (please encircle 1 2 3 4 5 )
My interaction partner seemed to find our interaction stimulating. (please encircle 1 2 3 4 5 )
My interaction partner communicated coldness rather than warmth. (please encircle 1 2 3 4 5 )
My interaction partner created a sense of distance between us. (please encircle 1 2 3 4 5 )
My interaction partner seemed detached during our interaction. (please encircle 1 2 3 4 5 )
My interaction partner was unwilling to share personal information with me. (please encircle 1 2 3 4 5 )
My interaction partner made our conversation seem intimate. (please encircle 1 2 3 4 5 )
My interaction partner created a sense of distance between us (this item is identical to the fourth item of this scale, perhaps by mistake). (please encircle 1 2 3 4 5 )
My interaction partner created a sense of closeness between us. (please encircle 1 2 3 4 5 )
My interaction partner acted bored by our conversation. (please encircle 1 2 3 4 5 )
My interaction partner was interested in talking to me. (please encircle 1 2 3 4 5 )
My interaction partner showed enthusiasm while talking to me. (please encircle 1 2 3 4 5 )
Items assessing self-reported copresence:
I did not want a deeper relationship with my interaction partner. (please encircle 1 2 3 4 5 )
I wanted to maintain a sense of distance between us. (please encircle 1 2 3 4 5 )
I was unwilling to share personal information with my interaction partner. (please encircle 1 2 3 4 5 )
I wanted to make the conversation more intimate. (please encircle 1 2 3 4 5 )
I tried to create a sense of closeness between us. (please encircle 1 2 3 4 5 )
I was interested in talking to my interaction partner. (please encircle 1 2 3 4 5 )
B2: SCHROEDER ET AL. QUESTIONNAIRE
All items are rated on a 5-point scale: (1= to a very small extent, 5= to a very high extent).
Items assessing collaboration:
To what extent did you experience that you and your partner collaborated? (please encircle 1 2 3 4 5 )
Think of some previous time (before today) when you enjoyed collaborating with someone. To what extent did you enjoy collaborating with your partner in today’s task? (please encircle 1 2 3 4 5 )
To what extent would you, on another occasion, like to carry out a similar task with your partner? (please encircle 1 2 3 4 5 )
Items assessing contribution to the task:
How would you estimate your and your partner’s share in solving the task? (please encircle 1 2 3 4 5 )
To what extent did you and your partner contribute to ….. (specific task)? (please encircle 1 2 3 4 5 )
Who talked the most, you or your partner? (please encircle 1 2 3 4 5 )
Items assessing presence:
To what extent did you have the experience of being in the same room as the cubes? (please encircle 1 2 3 4 5 )
When you think back on the task, to what extent can you have the experience right now that you are moving around in the room where the cubes were? (please encircle 1 2 3 4 5 )
To what extent did you experience the environment as a place you visited rather than something that you were looking at? (please encircle 1 2 3 4 5 )
Items assessing copresence:
To what extent did you have a sense of being in the same room as your partner? (please encircle 1 2 3 4 5 )
When you continue to think back on the task, to what extent do you have a sense that you are together with your partner in the same room?’’. (please encircle 1 2 3 4 5 )
Heuristic Evaluation VR (expert)
This is not a user test, but an expert review. You perform a Heuristic Evaluation with 4-6 people from your team or 4-6 designers-developers from other teams. The idea is that each of you will take the headset and go through the experience. After this, each of you will list challenges & problems that you have experiences for each heuristic (so Natural Engagement can show a list of multiple challenges that need to be solved, and then the next heuristic as well).
When everyone has created their list of issues, problems and challenges, you will start discuss each issues and then rate each one of them. (1= simple problem, easy solvable within project - 5= complex problem, unsolvable within project). You will create a Heuristic Evaluation Matrix to receive an overview of the amount and complexity of usability problems in total.
Our heuristics were motivated by the different nature of VEs, in particular, the need for intuitive interaction and the sense of immersion, which is important for many VR applications that aim to simulate reality as faithfully as possible (Stone, 2002).
The heuristics for VR
Tip: try to come up with some of your own heuristics, specified for your project.
Natural engagement. Interaction should approach the user’s expectation of interaction in the real world as far as possible. Ideally, the user should be unaware that the reality is virtual. Interpreting this heuristic will depend on the naturalness requirement and the user’s sense of presence and engagement.
Compatibility with the user’s task and domain. The VE and behaviour of objects should correspond as closely as possible to the user’s expectation of real world objects; their behaviour; and affordances for task action.
Natural expression of action. The representation of the self/presence in the VE should allow the user to act and explore in a natural manner and not restrict normal physical actions. This design quality may be limited by the available devices. If haptic feedback is absent, natural expression inevitably suffers.
Close coordination of action and representation. The representation of the self/ presence and behaviour manifest in the VE should be faithful to the user’s actions. Response time between user movement and update of the VE display should be less than 200 ms to avoid motion sickness problems.
Realistic feedback. The effect of the user’s actions on virtual world objects should be immediately visible and conform to the laws of physics and the user’s perceptual expectations.
Faithful viewpoints. The visual representation of the virtual world should map to the user’s normal perception, and the viewpoint change by head movement should be rendered without delay.
Navigation and orientation support. The users should always be able to find where they are in the VE and return to known, preset positions. Unnatural actions such as fly-through surfaces may help but these have to be judged in a trade-off with naturalness (see heuristics 1 and 2).
Clear entry and exit points. The means of entering and exiting from a virtual world should be clearly communicated.
Consistent departures. When design compromises are used they should be consistent and clearly marked, e.g. cross-modal substitution and power actions for navigation.
Support for learning. Active objects should be cued and if necessary explain themselves to promote learning of VEs.
Clear turn-taking. Where system initiative is used it should be clearly signalled and conventions established for turn-taking.
Sense of presence. The user’s perception of engagement and being in a ‘real’ world should be as natural as possible.
…….. you can create some of your own heuristics as well.
https://www.irit.fr/recherches/ICS/projects/twintide/upload/451.pdf https://www.researchgate.net/publication/222086342_Heuristic_evaluation_of_virtual_reality_applications
Last updated
