The goal of the study

At the start of a study, its goals must be clear. The goals drive most of the other elements of the framework.

Goals can be anything from all-encompassing (wanting a general feel of how usable a website is) to very specific (wanting to evaluate a new feature and how the users like it, or why the conversion rate went down after a website redesign). At this point in the study, it is more important to ensure that the goals are realistic (that is, that they can actually be achieved with a usability study) than very detailed.

For example, if the goal is to evaluate whether a new marketplace website can go live in its current state, then the goal is not feasible for a usability study: the study can only provide results about the usability of the website, and while we absolutely agree that usability should be a strong deciding factor, a go-live decision for a marketplace website should also be based on whether the product catalog is complete and relevant to customers, for example. The role of the UX researcher is to educate the stakeholders as to what can actually be achieved with a usability study and to get a consensus on that. The study will then be tailored to achieve those goals. It is imperative that everyone involved understands and agrees with the goals.

It is also important to document the goals so that all subsequent steps can refer back to them. It sets the focus for the study, and, while designing the discussion guide/script, the UX researcher should regularly refer back to the goal, in order to validate that the discussion guide/script is on track and hasn't wandered off. When writing up the report, we recommend always referring back to the original goal and documenting whether it has been refuted or validated, based on the results of the study.

The high-level goal of the study has to be unambiguous; details, however, can be flushed out when designing the discussion guide/script. The goal can also be comprised of multiple goals, such as testing a new feature while also testing possible icons that will be used to link to the feature.

The following sections will cover some goals that we have encountered during our usability studies, and how we rate the suitability of each.