Asking questions is a key activity in user research. This can, e.g., take place in “stand alone” form of an online survey or in combination with other approaches, such as an interview before conducting a usability test. A well-crafted inquiry can provide valuable clues for creating user-friendly solutions.
Sometimes, asking questions is regarded as an “easy” approach because it seemingly does not take much effort for preparation and does not eat up many resources. In the worst case, this impression can result in inquiries of low quality, which, in turn, provide low quality data.
The following article provides some hints on how to create inquiries that are beneficial for UX design projects.
What Do I want to Know?
I might sound trivial, but before creating questions, one should carefully define what one wants to learn. Spontaneously assembling questions can result in the focus shifting from what one initially wanted to learn to collecting questions that are easy to create, but which, in the worst case, only scratch the surface of the topic in question. This can even go as far as collecting questions that have little to nothing to do with the topic of interest. In these cases, the list of questions can become voluminous, without this resulting in a similar increase in insights gained.
It’s relatively easy to assemble a large set of questions aimed at eliciting easily accessible characteristics such as gender, age, hobbies, profession etc. But one should not assume that a sufficiently large set of these questions will necessarily provide deep insights regarding the personality of the person in question. When profound understanding of human beings is the goal, creating appropriate questions can become a complex task. One may also come to the conclusion, that asking questions is not the right approach to gain the desired insights.
Therefore, it should be clear for each and every question which insights they should provide in regard to the topic of interest.
Should One be Asking Questions in the First Place?
In some cases, reflecting insights to be gained can result in the decision not to ask questions at all. Sometime, e.g., insights are more easily gained through observation, as with people’s unconscious behaviors, which they cannot report when asked about them. Certain topics can be uncomfortable for people to talk about. In those cases, too, other methods if inquiry than asking direct questions can be more appropriate.
If, e.g., the goal is to gain insights regarding the traits openness to experience and conscientiousness, simply asking about those traits might only yield very limited insight. (“How open are you to experience?” “How conscientious are you?”). A more appropriate approach might be observing behavior in the relevant situations and making inferences regarding the traits in question without having data pass a subjective “filter” of the person first.
Questions Have to Be Understandable
Another property of good questions is that they are clearly understandable for the target group. Good questions are user-centered. Again, this is sometimes less trivial than it seems. Of course, the people asking the questions usually know what they are aiming for. However, this does not necessarily mean that it also holds true for the people being asked. In the “best” case, people do not understand the meaning of a question and provide feedback on that fact. In the worst case, people misunderstand the question without the interviewer realizing that this is the case. This can result in the interviewees answering another question (meaning) that has not been asked for without the interviewer realizing that this is the case.
An interviewer might, e.g., inquire about someone’s opinion regarding artificial intelligence and have a very elaborate concept of AI with its possibilities and limitation in mind. Someone with limited expertise and mainly influenced by the depiction of AI in popular culture may use this personal perspective as reference. This means that both “frames of reference” differ considerably without one or both parties necessarily being aware of that. In consequence, the interpretation of answers may be skewed.
Therefore, questions should be tested, similar to usability testing. This can be done, e.g., by asking representative interviewees the questions and letting them provide feedback on what they think the questions are aiming at. This approach can uncover misunderstandings, which provides the basis for adapting and optimizing the questions before the real interview is taking place.
Open Questions
Regarding the effort that has to be invested in analyzing results, it can be tempting to only use closed questions, i.e. questions for which the interviewee can choose from a fixed set of answer options, such as “yes” or “no”. The downside often is that the potential for new insights is limited, because the interviewee cannot elaborate why an answer has been chosen. This is especially the case for written surveys which are administered without direct contact.
It may, e.g., be flattering to receive “Yes” as an answer when asking “Do you prefer our product over that of our competitor?”, but in the end, this only allows for a description of the status quo without further insights.
In case of an actual interview, having the interviewee choose from a fixed set of answering options can feel awkward and it can be difficult to establish a natural flow of interaction between interviewer and interviewee.
It is therefore recommendable to also use open questions in order to elicit more in-depth insights and also to have a more natural flow of conversation that does not simply consist of highly-frequent question-answer “ping pong”.
Not Only Asking, But Listening
For actual interviews, it is crucial to not only ask good questions, but also attentively listening to the interviewee’s answers. It can, e.g., be helpful to paraphrase answers that are given. This means that the interviewer describes in their own words their understanding of the answer. This is helpful feedback for the interviewee that shows that the answer has been “received” and understood. In addition, attentively listening can highlight options for going into more depth, when the answers contain interesting “hints”. This approach can easily be integrated in semi-structured interviews, in which interviewers can adapt and supplement a core set of pre-defined questions with additional questions as they see fit.
With this approach, one may, e.g., discover, that the judgment of a software is not influenced by (new) technical features, but rather by (changing) life circumstances. Not exploring those with new/modified questions would leave potential relevant insights undiscovered.
An interview should not simply mean “ticking off questions from a list”, but rather be tailored to the specific interviewee.
Conclusion
Asking questions is an important activity in user research. One should not, however, regard interviews as something that can quickly and easily be planned and realized. Questions should be crafted carefully and be tested before they are used for gathering data. In an interview, the interviewer should listen attentively and also check whether it is possible and desirable to customize the interview to the concrete interviewee. By keeping such quality criteria of good questions and interviews in mind, one can increase the chances of eliciting valuable information that are beneficial for UX design projects.