HCDE 517: Usability Studies (Winter 2022)

Teaching Staff & Office Hours

Sean Munson
smunson at uw dot edu
Office hours: by appointment

Laura Swartley
swartl at uw dot edu
Office hours: by appointment

Course Description

HCDE 517 will introduce you to usability testing and to usability research as a user-centered design strategy. The course takes a process approach; you will learn how to define your audiences and issues, create investigative procedures that answer your questions, administer the procedures, analyze the results, and report your findings effectively.

Though we use primarily examples of computational systems in this course, many of the methods, strategies, and approaches apply to products in various media (software, hardware, or services). You are encouraged to bring a variety of examples into the class.

It is important that you keep up with the assignments and that you seek help — from the teaching team, your teammates, or your classmates — if you are struggling.

Learning Objectives

At the end of HCDE 517, you should be able to:


Slides and notes

Slides/notes handouts from the lectures: Generally made available via the Canvas site (under files), after our class meeting.


Unless otherwise announced, papers will be posted on the Canvas site and/or linked to from this syllabus.


There are two required textbooks:

Both should be available in the UW Bookstore.

Course discussion site

One of the best ways to get help during the course will be via the course Slack workspace we have created. You are encouraged to respond to (comment on) other students' messages. Don't worry about giving wrong information: the instructors, or your peers, will post corrections if necessary.

If you have something of a private nature that you don't want to share with the other students, feel free to send a message to one of us individually.

Class policies

Collaboration, academic honesty, and integrity

This quarter, you will be asked to contribute to the success of your colleagues' projects by providing them with feedback on their works-in-progress. Your participation in workshop activities will be graded for the quality, usefulness, and insightfulness of your comments. You will also be collaborating with colleagues on assignments.

If you receive assistance on an individual assignment, please indicate the nature and the amount of assistance you received. If you have more experience with some of the material and are willing to help other students, please feel free to do so. Just remember that your goal is to help teach the material to the classmate receiving the help.

The essence of academic life revolves around respect not only for the ideas of others, but also their rights to those ideas and their circulation. It is essential to take the utmost care that the ideas and expressions of ideas of other people always be appropriately attributed, and, where necessary, cited. For writing assignments and anywhere in print, when ideas or materials of others are used, they must be cited. Any direct quotes must be cited. In any situation, if you have a question, please ask. Such attention to ideas and acknowledgment of their sources is central not only to academic life, but life in general.

Course conduct is governed by the University of Washington Student Conduct Code and the College of Engineering Academic Misconduct Policy. You should also acquaint yourself with the HCDE Plagiarism Policy. Failure to properly cite or attribute an idea will result in a zero on that assignment and a report to the College of Engineering; severe or repetitive instances may result in more severe consequences.

To reiterate, the collaboration policy is as follows. Collaboration in the class is encouraged — you can get help from anyone as long as it is clearly acknowledged. Use of work — yours or others', from this course or similar courses — from previous quarters is not allowed. The authorship of any assignments must be in your own style and done by you, even if you get help. Any significant help must be acknowledged in writing; any use of others' words or ideas must be cited and attributed appropriately.

Classroom Rules

I like a relaxed classroom where everyone feels comfortable. You are welcome to bring drinks or snacks to class (though, due to COVID restrictions, will need to eat during breaks). You can bring your laptop to class. I would rather have you come to class and listen with one ear than not come to class at all. As a courtesy to others, be sure to put your phone on silence/vibrate. Coming late to and/or leaving early from class is okay, as long as you don't disturb your classmates. Ask questions at any time, and if you have some expertise in a particular topic, please to raise you hand and share it with the class. If you aren't comfortable doing so, you should also feel free to post it to the Slack workspace!

Our primary purpose in the classroom is to interact and learn from each other. While non-participation (sleeping; checking email) is permitted, we urge you to consider its impact on your fellow students. Class is a lot more fun when everyone is engaged; a few visibly disengaged people can suck the energy out of a room. We reserve the right to ask you a question to try to reengage you if you seem to have checked out, but will try not to do this in an embarrassing way.

Contacting the instructors

Our prefered method of contact (for quick questions, to let us know you will miss class, to schedule a meeting, etc.) is Slack or email. I (Sean) don't check my voicemail or Canvas messages regularly. When emailing:

I want to emphasize that your feedback is welcome. I may adjust the course based on what you say, I may save your comments for next year, or I may disagree, but in any case, your thoughts are appreciated.

Student rights

Please read the HCDE statement on student rights.

Acccess & Accommodations

Your experience in this class is important to us. If you have already established accommodations with Disability Resources for Students (DRS), please communicate your approved accommodations to us at your earliest convenience so we can discuss your needs in this course.

If you have not yet established services through DRS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but not limited to; mental health, attention-related, learning, vision, hearing, physical or health impacts), please contact DRS at 206.543.8924, uwdrs@uw.edu, or disability.uw.edu. DRS offers resources and coordinates reasonable accommodations for students with disabilities and/or temporary health conditions.

Reasonable accommodations are established through an interactive process between you, your instructor(s), and DRS. It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law.

Religious Accommodations

Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities.

The UW’s policy, including more information about how to request an accommodation, is available. Accommodations must be requested within the first two weeks of this course using the request form.

Student Conduct

The University of Washington Student Conduct Code (WAC 478-121) defines prohibited academic and behavioral conduct and describes how the University holds students accountable as they pursue their academic goals. Allegations of misconduct by students may be referred to the appropriate campus office for investigation and resolution.

Harassment will not be tolerated, and consequences will include immediate removal from the Slack workspace and other course tools. I also emphasize the importance of respecting your classmates' (and teaching team's) boundaries around how and when they wish to be contacted.

I am not an official required to report under UW's Title IX plan. This means I am not legally required to report misconduct or crime; if you share a problem with me, I will work with you to determine next steps. An exception is if anyone involved is under 18. You also can reach out to UW confidential advocates or SafeCampus.

Work in the Course — Getting a Grade

Each assignment is designed to test your achievement against one or more of the learning objectives. Different assignments emphasize different learning objectives. Please note that some grading will be subjective in nature.

There are three categories of assignments:

In addition, you will complete a final self and team evaluation report, reflecting on your learning and participation across the quarter.

Late assignments

It your responsibility to make sure that each assignment has been turned in on time. Except by prior arrangement, we do not grade late assignments, though we will return them with feedback. For personal emergencies, we will work out an alternative plan with you and your team if you contact us before the due date.

Quality of Communication Assignments

All written assignments for this class must be of high quality: thoroughly proofread, well organized, and stylistically appropriate for the context. If in doubt, err on the more formal, polished, professional side. Writing quality will be a graded component of every written assignment, presentation clarity and engagement will affect your grades on all presentation assignments. In the workplace, if your results or ideas are of high quality but your communication is not, no one will be able to benefit from the work you do.


The graded work in the course will be weighted roughly as follows to determine a final percentage grade.

Reading presentation & discussion10%
P1. Preliminary proposalCredit/No credit
P2. Interaction map10%
P3. Usability study plan15%
P4. Usability study kit15%
P5. Study results (divided among presentation & report)35%
Self + Team-evaluation Report10%
Class participation± 5%*

* possibly more for particularly constructive participation. Attendance is not graded, but it is a lot easier to participate if you are here.

Also see the HCDE grading policy.

Course Outline

In this course, we will cover usability studies, including their role in the design process, what questions they can answer, and some of the different evaluations and tests you might peform. We will discuss how to plan and conduct usability tests, and how to report results from usability studies. To practice these skills, we will use in-class workshops and an accompanying, quarter-long project.

Success in the Course

From my prior experience teaching HCDE 517, the most common places students run into trouble are: (1) not starting soon enough, (2) letting team issues fester, and (3) not negotiating a clear and reasonable scope with clients at the start.

The most common pitfall is not starting on any portion of the project soon enough. You'll be working with participants and (possibly) real world clients, and with technology that may have issues. Recruiting almost always takes longer than you think, people may not show up, or there may be a chance snow storm. The website you are studying might go down for maintenance. The app you are evaluting might auto-update to a new version, ruining the script for your evaluation. If you start early, you'll have time to manage these setbacks. If you don't, you won't.

Additionally, as relative newcomers to this material, you tend to need some time with your data (especially any qualitative data) to draw out a strong analysis and good discussion. Doing a study at last minute and then rushing your writeup and analysis tends to have poor outcomes.

Second, you'll be working in teams and it is important to attend to team dynamics. Early in the quarter, you should have a discussion in your team in which you review and negotiate communication expectations and your goals for the course and project. As the quarter progresses, check in, see how things are going, and adjust as necessary.

Good luck and welcome aboard!

Course Schedule and Important Dates (view on separate page)

This is tentative. It may change based on your interests, class discussion, or guest speakers' availability.
WeekDatesTopicReadingsAssignments due
1 4 January

Course overview

  • Introductions
  • What is usability?
  • History of usability?
  • Learning & Memory


2 11 January

Planning & considerations for usability testing

  • Why do we need usability testing?
  • Usability testing within the UCD process
  • Identifying usability concerns
  • Usability inspection methods

Choose teams + Projects

Workshop: Brainstorming usability concerns

From the books (required)

  • Tullis & Albert, “Introduction”
  • Rubin & Chisnell, Chapters 1 -3



Extend topic: Heuristic evals for specific settings

Extend topic: Cognitive Walkthroughs in other Settings

Aaron R. Lyon, Jessica Coifman, Heather Cook, Erin McRee, Freda F. Liu, Kristy Ludwig, Shannon Dorsey, Kelly Koerner, Sean A. Munson & Elizabeth McCauley The Cognitive Walkthrough for Implementation Strategies (CWIS): a pragmatic method for assessing implementation strategy usability. Implementation Science Communications 2 (2021).

Read project descriptions

Sign up for reading presentation

3 18 January

Planning for a usability study, cont.

  • More methods for identifying usability issues
  • Selecting representative users

From the books (required)

  • Rubin & Chisnell, “Develop the Test Plan” (ch 5) & “Set Up a Testing Environment” (ch 6)
  • Tullis & Albert, “Background” (ch 2)



Extend topic: Accessibility

P1. Preliminary proposal

4 25 January

Forming an initial test plan

  • What are the responsibilities of a tester?
  • What are general considerations to drive a test plan?
  • What is essential and what is optional when preparing a meaningful study?

From the books (required)

  • Rubin & Chisnell, “Skills for Test Moderators” (ch 4) & “Find and Select Participants” (ch 7)
  • Tullis & Albert, “Planning a Usability Study” (ch 3)



Extend topic: Usability, children, and aging

Extend topic: Usability and automation

P2. Interaction map
  • Draft: in class
  • Final: Sunday, 30 Jan
5 1 February

Preparing a test with measurable results

  • What test materials do we need for a usability study?
  • What will be measured?
  • How do we create successful data collection instruments?

Workshop: Study plans

From the books (required)

  • Rubin & Chisnell, “Prepare Test Materials” (ch 8).
  • Tullis & Albert, “Performance Metrics” (ch 4), “Issues-based Metrics” (ch 5), “Self-reported Metrics” (ch 6).



Extend topic: The Hamburger Icon

P3. Study plan

  • Draft in class
  • Final by Sunday, 6 Feb
6 8 February

Conducting a usability study

  • What is “thinking aloud”?
  • What is key to successful facilitation?
  • How do we collect data from a usability study?

Workshop: Study kits

From the books (required)

  • Rubin & Chisnell, “Conduct the Test Sessions” (ch 9).
  • Tullis & Albert, “Behavioral and Physiological Metrics” (ch 7).



Extend Topic: Invisible OSs, Recognition, and Recall

P4. Study kit

  • Draft: In class
  • Final: Sunday, 12 Feb
7 15 February

Data analysis & reporting

  • Validity and Reliability
  • Analysis and Reporting

Workshop: Data analysis

From the books (required)

  • Rubin & Chisnell, “Analyze Data and Observations” (ch 11).
  • Tullis & Albert, “Combined and Comparative Metrics” (ch 8).



Extend topic: Voice UI

    Extend topic: Large data in usability

    8 22 February

    Presenting your findings

    • Discuss details for the final assignments
    • How can data and findings be meaningfully shaped for stakeholders?
    • Reporting and communicating results

    Usability experiences in sociotechnical systems, can things be “too” usable? (& other dilemmas)

    From the books (required)

    • Rubin & Chisnell, “Report Findings and Recommendations” (ch 12).



    Extend topic: Technical choices that affect usability

    9 1 March

    Guest lecture: (Tentative - TBD)

    Reporting Examples

    Usability in the field, and moving forward

    • What are the benefits & challenges of conducting usability activities in the field?
    • What are other tools and approaches?


    Extend topic: Biometrics

    10 8 March

    Course wrap up & Presentations



    In this version of HCDE517, I extensively draw from and/on materials developed by Rebecca Destello, Andy Davidson, and Judy Ramey.