Survey standard techniques for preparing questionnaires for evaluating user satisfaction
HCI literature demonstrates concerns that existing satisfaction rating scales may inadequately measure end user satisfaction (Gitte and Cathy, 2003). The reason for this lies on the difficulty to take under consideration the psychology of human satisfaction and the individual biological and sociological factors that drive emotions and behavior. User satisfaction is well related to usability and efficiency but, according to study findings, also relates to user’s attitude towards computers, user computer literacy and user age. Findings also show a significant relationship between the object of use of the system and user anticipations. User preconceptions and prior experiences are negatively related with end user satisfaction.
Well established techniques, such as the System Usability Scale (SUS), the Software Usability Measurement Inventory (SUMI) and the Questionnaire for User Interaction Satisfaction (QUIS) focus on International Standards for HCI and usability, primarily concerned with the effectiveness, efficiency, comfort and acceptability of use. During the development of the above recognized methods of testing user satisfaction, ergonomics of HCI, as managed by the ISO committee, were put into consideration (Bevan, 2000). Surveys for evaluating a system have been developed, validated and standardized by enabling measurement of requirements expressed by European directives (90/270/EEC) and ISO (ISO 9241).
The design of a survey encompasses several interrelated metrics as well as methods and concepts to obtain the most complete and accurate information possible.
The effectiveness of a questionnaire will depend on the type of collected information and thus, on the researcher objectives. If the purpose of the questionnaire is to collect qualitative results, a better understanding of the user experience is to be expected. Quantitative results, on the other hand, will test specific and measurable cases and may restrict the outcome of the research.
A good questionnaire is well organized, easily comprehensible, adapts to individual cultural needs (question wording, appropriate language), takes into account demographic factors (such as age, education), encourages participation and provides the respondents with an easy method of providing an answer. Another important design consideration is the construction of the survey questions that must not “suggest” a narrative and influence the correspondent but allow for the most possible unbiased results. The appearance of the questionnaire can have a significant effect on the outcome of the survey. A creative use of space and typeface and the use of colors where possible, will make it easier and more pleasant for the correspondents to fill in the questions. Last, having the questions reviewed may determine the feasibility of the evaluation and the quality of the final results.
Gitte, L., Cathy, D., 2003. What is this evasive beast we call user satisfaction? Interacting with Computers, 15(3), pp.429-52. Available through: ScienceDirect.com website <http://www.sciencedirect.com/science/article/pii/S0953543802000632>
Bevan, N., 2000. ISO And Industry Standards for User Centered Design. [online via UsabilityNet.org] Serco Usability Services. Available at: <http://www.usabilitynet.org/trump/documents/Usability_standards.ppt.pdf>