This document is intended as a high-level overview of the concepts, methods, benefits, and challenges of user research as a discipline within an Information Technology organization.
user_research_advisory.pdf | 287 KB |
user_research_advisory.pdf | 287 KB |
Authors |
|
---|---|
Version | 1.0 |
Last Revised | 3-Feb-2022 |
Status | Released |
Document Type | Single Topic Guidance |
Audience Level |
|
This document is intended as a high-level overview of the concepts, methods, benefits, and challenges of user research as a discipline within an Information Technology organization. This advisory encourages individual contributors involved in the discovery, design, development, and implementation of new products, processes, or services; or the monitoring of success criteria for existing solutions, to incorporate appropriate research methods as part of their regular strategic and operational practices. Links to additional resources, including guides for performing select user research methods, are provided.
Understanding the human dimensions that shape a person’s interactions with an IT organization—and the solutions it delivers—are critical for the successful delivery of value to the people it ultimately serves. Integrating user research methods with routine IT procedural activities (e.g. requirements gathering, solution design, user acceptance testing) enables the organization to:
Potential user research methods that supplement existing IT activities include user interviews, surveys, expert reviews, usability tests, and the collection of experience metrics through usage analytics.
The proposed user research methods provide opportunities for participation from everyone in the IT organization, ensuring that the organization and its individual contributors:
There are many opportunities for user research to augment and improve the successful delivery of solutions. While many User Experience methods and techniques are focused on gathering insights of the human impact of technology decisions, a few recommended methods should be used by technologists to develop a richer understanding of their end-users.
The inclusion of one or more of the following research methods into routine IT practices will assist with defining and developing new solutions, as well as the evaluation and improvement of existing technologies and services.
Traditional business activity to supplement: Discovery and requirements gathering activities
Interviews with representative end-users, as well as business stakeholders, are an invaluable source for gathering information about people’s needs and goals; their motivations, thoughts, and decision-making processes; and their attitudes and beliefs. The User Interview method lends itself to allowing broad participation from technology teams and business stakeholders, as a way of information gathering and empathy building (i.e. hearing and observing first-hand from people is a natural and effective way to make connections with the people who utilize your products and services).
It is also important to note that interviewing business stakeholders as part of a user interview process is highly recommended. Knowing and understanding the business goals more deeply, as well as its motivations and expectations, will help find gaps and overlaps between business and user needs, provide a path to stakeholder alignment, and assist in the process of better defining problems and imagining solutions.
IMPORTANT: User Interviews are not the same as requirements gathering. Although requirements gathering is a traditional way for technologists to assess user needs, the process tends to focus more on the capabilities and attributes of a proposed solution, as opposed to exploring and evaluating the human dimensions of cognition, emotion, and behavior; which can lead to greater insights for producing innovative and alternative solutions.
Traditional business activities to supplement: Discovery and requirements gathering activities
Surveys are another way of gathering information about users that can reveal similar human dimensions as User Interviews for research focused on new and existing solutions, with the added benefit of scalability, which can help to surface the size or importance of a particular aspect. However, due to their one-way nature, surveys limit the depth to which the researcher can probe on areas of interest, and don’t elicit the level of empathy that a two-way conversation can produce.
Traditional business processes to supplement: Design and development, Implementation, Quality Assurance (QA), User Acceptance Testing (UAT)
An effective method for identifying baseline usability issues is to have an IT professional trained in this method or a User Experience practitioner perform an Expert Review, sometimes referred to as a Heuristic Review—a systematic review of a solution, which looks to identify any violations of common UX principles and best practices. This method can surface critical issues, but its effectiveness is limited by the reviewer’s knowledge of usability best-practices, and the context and subject-matter involved with the solution.
Traditional business process to supplement: Design and development, Implementation, Quality Assurance (QA), User Acceptance Testing (UAT)
Testing concepts, prototypes, and developed solutions can provide information about how a solution performs, and whether it is an adequate fit-to-need for intended users. These tests can have a quantitative dimension—measuring what users do, such as how long it takes to complete a task, and what kinds of user errors are repeated (e.g. navigation patterns that cause a user to become ‘lost)’—as well as qualitative dimensions, which help the technologist to better understand user comprehension and why users behave in certain ways.
IMPORTANT: Usability testing is not Quality Assurance (QA) or User Acceptance Testing (UAT). QA and UAT can surface issues around whether a proposed solution meets necessary technical aspects, and answer questions such as ‘Did we build the thing right?’, as opposed to the more holistic, user-centered approach, which can get the technologist closer to understanding “Did we build the right thing?”
Traditional business process to supplement: Monitoring of success criteria and KPIs
Measuring the utilization of a current service or product with the additional of experience metrics can reveal important insights into how effective and efficient the solution is at serving both user and business goals.
For IT solutions to be utilized and return value to the organization, it is critical that the human dimensions of end-users—cognition, behavior, and context—be placed at the center of design, development, and decision-making processes.
Uncovering these human insights helps improve both strategic and tactical decisions by rooting them in empathy and understanding for the people who will ultimately use and benefit from a product or service; helping to mitigate the pitfalls of assumption and bias, by relying on observation rather than anecdote and preconceived beliefs.
By refocusing success from the successful deployment of solutions to a more user-centered, empathetic lens focused on the human dimensions of a solution, the IT organization can better plan for what and how it delivers value to its customers. User Research is the necessary, evidence-based means for uncovering the human insights that make empathetic, user-centered decision-making possible.
User Research can be roughly divided into two lines of inquiry: one that looks to design or find new solutions and opportunities, and one that looks to evaluate existing solutions.
Within both of these general branches of research are further categories of research types that can help us to understand different dimensions of the human experience:
There are many User Research methods, each of which has strengths depending on the research goals of the inquiry.
Some common methods and their fit for research objectives are listed below.
Generative | Evaluative | Attitudinal | Behavioral | Qualitative | Quantitative | |
---|---|---|---|---|---|---|
Usability Testing | X | X | X | X | X | X |
Focus Groups | X | X | X | X | ||
User Interviews | X | X | X | X | ||
Contextual Inquiry | X | X | X | X | X | |
Card Sorting | X | X | X | X | X | |
Surveys | X | X | X | X | ||
Click-stream Analysis | X | X | X | |||
Expert Review | X | |||||
Usage Analytics | X | X | X |
Every role in an Information Technology organization provides inputs that impact the overall experience of products and services, including the process by which a user realizes the value of a solution, its fit-to-need, its completeness and accuracy, and its utilization. Therefore, everyone would benefit from a deeper understanding of the effects their contributions play.
The level of effort and expertise involved in research methods varies, but a broad set of opportunities for participation by anyone in an IT organization include:
Including everyone in the User Research process helps to ensure that the organization doesn’t lose sight of who it serves and why it matters.
Ideally, User Research methods would be employed at two key points in an IT organization:
In addition, User Research should be performed whenever there are questions or assumptions about user needs or behaviors driving IT decision making, or if the human impact of decisions is not well understood.
The need for User Research exists along the entire IT lifecycle, from strategic planning to the design, development, and procurement of new solutions, to the operational maintenance and ultimate decommissioning of existing products and services.
At a minimum, User Research activities should be incorporated in an integrated way into planned activities surrounding:
While User Research should be an ongoing process embedded in every stage of the IT lifecycle, most individual methods applied to specific research goals will eventually reach a point at which new information is no longer providing meaningful insights for the research objective(s). These diminishing returns can generally be mitigated at the practical level by:
However, the answer to how much User Research should be conducted for any initiative at the strategic level should take into consideration:
By layering an understanding of the human dimensions of cognition, behavior, and context to IT activities, User Research helps IT organizations create better outcomes for the people it serves. User Research supports the organization in its strategy and decision-making by revealing new opportunities, improving decision making, and reducing uncertainty—through the collection of evidence, and testing and validation of assumptions and hypotheses—allowing it to make decisions more confidently and efficiently regarding the value and the impact of its activities.
The following resources are intended as a starting point for learning and putting into practice the UX methods described in this advisory.