HCDE 517: Usability Studies (Winter 2018)

Teaching Staff & Office Hours


Sean Munson
smunson at uw dot edu
Office hours: by appointment


Brook Sattler
brooks2 at uw dot edu
Office hours: by appointment

Course Description

HCDE 517 will introduce you to usability testing and to usability research as a user-centered design strategy. The course takes a process approach; you will learn how to define your audiences and issues, create investigative procedures that answer your questions, administer the procedures, analyze the results, and report your findings effectively.

Though we use primarily examples of computational systems in this course, many of the methods, strategies, and approaches apply to products in various media (software, hardware, or services). You are encouraged to bring a variety of examples into the class.

It is important that you keep up with the assignments and that you seek help — from the teaching team, your teammates, or your classmates — if you are struggling.

Learning Objectives

At the end of HCDE 517, you should be able to:

Resources

Slides and notes

Slides/notes handouts from the lectures: Generally made available via the Canvas site (under files), after our class meeting.

Papers

Unless otherwise announced, papers will be posted on the Canvas site and/or linked to from this syllabus.

Textbooks

There are two required textbooks:

Both should be available in the UW Bookstore.

Course discussion site

One of the best ways to get help during the course will be via the course Slack workspace we have created. You are encouraged to respond to (comment on) other students' messages. Don't worry about giving wrong information: the instructors, or your peers, will post corrections if necessary.

If you have something of a private nature that you don't want to share with the other students, feel free to send a message to one of us individually.

Class policies

Collaboration, academic honesty, and integrity

This quarter, you will be asked to contribute to the success of your colleagues' projects by providing them with feedback on their works-in-progress. Your participation in workshop activities will be graded for the quality, usefulness, and insightfulness of your comments. You will also be collaborating with colleagues on assignments.

If you receive assistance on an individual assignment, please indicate the nature and the amount of assistance you received. If you have more experience with some of the material and are willing to help other students, please feel free to do so. Just remember that your goal is to help teach the material to the classmate receiving the help.

The essence of academic life revolves around respect not only for the ideas of others, but also their rights to those ideas and their circulation. It is essential to take the utmost care that the ideas and expressions of ideas of other people always be appropriately attributed, and, where necessary, cited. For writing assignments and anywhere in print, when ideas or materials of others are used, they must be cited. Any direct quotes must be cited. In any situation, if you have a question, please ask. Such attention to ideas and acknowledgment of their sources is central not only to academic life, but life in general.

Course conduct is governed by the University of Washington Student Conduct Code and the College of Engineering Academic Misconduct Policy. You should also acquaint yourself with the HCDE Plagiarism Policy. Failure to properly cite or attribute an idea will result in a zero on that assignment and a report to the College of Engineering; severe or repetitive instances may result in more severe consequences.

To reiterate, the collaboration policy is as follows. Collaboration in the class is encouraged — you can get help from anyone as long as it is clearly acknowledged. Use of work — yours or others', from this course or similar courses — from previous quarters is not allowed. The authorship of any assignments must be in your own style and done by you, even if you get help. Any significant help must be acknowledged in writing; any use of others' words or ideas must be cited and attributed appropriately.

Classroom Rules

I like a relaxed classroom where everyone feels comfortable. You are welcome to bring drinks or snacks to class (assuming it is allowed in the room we are in). You can bring your laptop to class. I would rather have you come to class and listen with one ear than not come to class at all. As a courtesy to others, be sure to put your phone on silence/vibrate. Coming late to and/or leaving early from class is okay, as long as you don't disturb your classmates. Ask questions at any time, and if you have some expertise in a particular topic, please to raise you hand and share it with the class. If you aren't comfortable doing so, you should also feel free to post it to the Slack workspace!

Our primary purpose in the classroom is to interact and learn from each other. While non-participation (sleeping; checking email) is permitted, we urge you to consider its impact on your fellow students. Class is a lot more fun when everyone is engaged; a few visibly disengaged people can suck the energy out of a room. We reserve the right to ask you a question to try to reengage you if you seem to have checked out, but will try not to do this in an embarrassing way.

Contacting the instructors

Our prefered method of contact (for quick questions, to let us know you will miss class, to schedule a meeting, etc.) is email. I (Sean) don't check my voicemail or Canvas messages regularly. When emailing:

I want to emphasize that your feedback is welcome. I may adjust the course based on what you say, I may save your comments for next year, or I may disagree, but in any case, your thoughts are appreciated.

Accommodations

If you think you need an accommodation for a disability, please let us know at your earliest convenience. Some aspects of this course, the assignments, the in-class activities, and the way we teach may be modified to facilitate your participation and progress. To request accommodations due to a disability, please contact the Disability Student Services Office at 206.543.6450 (voice), 206.543.6452 (TTY), 206.685.7264 (fax), or email at dso@u.washington.edu. If you have a letter from the DSS office indicating that you have a disability that requires academic accommodations, please present the letter to me so we can discuss the accommodations that you might need in this class.

Student rights

Please read the HCDE statement on student rights.

Work in the Course — Getting a Grade

Each assignment is designed to test your achievement against one or more of the learning objectives. Different assignments emphasize different learning objectives. Please note that some grading will be subjective in nature.

There are three categories of assignments:

In addition, you will complete a final self and team evaluation report, reflecting on your learning and participation across the quarter.

Late assignments

It your responsibility to make sure that each assignment has been turned in on time. Except by prior arrangement, we do not grade late assignments, though we will return them with feedback. For personal emergencies, we will work out an alternative plan with you and your team if you contact us before the due date.

Quality of Communication Assignments

All written assignments for this class must be of high quality: thoroughly proofread, well organized, and stylistically appropriate for the context. If in doubt, err on the more formal, polished, professional side. Writing quality will be a graded component of every written assignment, presentation clarity and engagement will affect your grades on all presentation assignments. In the workplace, if your results or ideas are of high quality but your communication is not, no one will be able to benefit from the work you do.

Grading

The graded work in the course will be weighted roughly as follows to determine a final percentage grade.

Reading presentation & discussion10%
P1. Preliminary proposalCredit/No credit
P2. Interaction map10%
P3. Usability study plan15%
P4. Usability study kit15%
P5. Study results (divided among presentation & report)35%
Self + Team-evaluation Report10%
Class participation± 5%*

* possibly more for particularly constructive participation. Attendance is not graded, but it is a lot easier to participate if you are here.

Also see the HCDE grading policy.

Course Outline

In this course, we will cover usability studies, including their role in the design process, what questions they can answer, and some of the different evaluations and tests you might peform. We will discuss how to plan and conduct usability tests, and how to report results from usability studies. To practice these skills, we will use in-class workshops and an accompanying, quarter-long project.

Success in the Course

From my prior experience teaching HCDE 517, the most common places students run into trouble are: (1) not starting soon enough, (2) letting team issues fester, and (3) not negotiating a clear and reasonable scope with clients at the start.

The most common pitfall is not starting on any portion of the project soon enough. You'll be working with participants and (possibly) real world clients, and with technology that may have issues. Recruiting almost always takes longer than you think, people may not show up, or there may be a chance snow storm. The website you are studying might go down for maintenance. The app you are evaluting might auto-update to a new version, ruining the script for your evaluation. If you start early, you'll have time to manage these setbacks. If you don't, you won't.

Additionally, as relative newcomers to this material, you tend to need some time with your data (especially any qualitative data) to draw out a strong analysis and good discussion. Doing a study at last minute and then rushing your writeup and analysis tends to have poor outcomes.

Second, you'll be working in teams and it is important to attend to team dynamics. Early in the quarter, you should have a discussion in your team in which you review and negotiate communication expectations and your goals for the course and project. As the quarter progresses, check in, see how things are going, and adjust as necessary.

Good luck and welcome aboard!

Course Schedule and Important Dates (view on separate page)

WeekDatesTopicReadingsAssignments due
1 4 January

Course overview

  • Introductions
  • What is usability?
  • History of usability?
  • Learning & Memory

Required

2 11 January

Planning & considerations for usability testing

  • Why do we need usability testing?
  • Usability testing within the UCD process
  • Identifying usability concerns
  • Usability inspection methods

Choose teams + Projects

Workshop: Brainstorming usability concerns

From the books (required)

  • Tullis & Albert, “Introduction”
  • Rubin & Chisnell, Chapters 1 -3

Required

Optional

Extend topic: Heuristic evals for specific settings

Read project descriptions

Sign up for reading presentation

3 18 January

Planning for a usability study, cont.

  • More methods for identifying usability issues
  • Selecting representative users

From the books (required)

  • Rubin & Chisnell, “Develop the Test Plan” (ch 5) & “Set Up a Testing Environment” (ch 6)
  • Tullis & Albert, “Background” (ch 2)

Required

Optional

Extend topic: Accessibility

P1. Preliminary proposal

4 25 January

Forming an initial test plan

  • What are the responsibilities of a tester?
  • What are general considerations to drive a test plan?
  • What is essential and what is optional when preparing a meaningful study?

From the books (required)

  • Rubin & Chisnell, “Skills for Test Moderators” (ch 4) & “Find and Select Participants” (ch 7)
  • Tullis & Albert, “Planning a Usability Study” (ch 3)

Required

Optional

Extend topic: Usability, children, and aging

Extend topic: Usability and automation

P2. Interaction map
  • Draft: in class
  • Final: Sunday, 28 Jan
5 1 February

Preparing a test with measurable results

  • What test materials do we need for a usability study?
  • What will be measured?
  • How do we create successful data collection instruments?

Workshop: Study plans

From the books (required)

  • Rubin & Chisnell, “Prepare Test Materials” (ch 8).
  • Tullis & Albert, “Performance Metrics” (ch 4), “Issues-based Metrics” (ch 5), “Self-reported Metrics” (ch 6).

Required

Optional

Extend topic: The Hamburger Icon

P3. Study plan

  • Draft in class
  • Final by Sunday, 4 Feb
6 8 February

Conducting a usability study

  • What is “thinking aloud”?
  • What is key to successful facilitation?
  • How do we collect data from a usability study?

Workshop: Study kits

From the books (required)

  • Rubin & Chisnell, “Conduct the Test Sessions” (ch 9).
  • Tullis & Albert, “Behavioral and Physiological Metrics” (ch 7).

Required

Optional

Extend topic: Additional Techniques

P4. Study kit

  • Draft: In class
  • Final: Sunday, 10 Feb
7 15 February

Data analysis & reporting

  • Validity and Reliability
  • Analysis and Reporting

Workshop: Data analysis

From the books (required)

  • Rubin & Chisnell, “Analyze Data and Observations” (ch 11).
  • Tullis & Albert, “Combined and Comparative Metrics” (ch 8).

Required

Optional

Extend Topic: Invisible OSs, Recognition, and Recall

8 22 February

Presenting your findings

  • Discuss details for the final assignments
  • How can data and findings be meaningfully shaped for stakeholders?
  • Reporting and communicating results

Usability experiences in sociotechnical systems, can things be “too” usable? (& other dilemmas)

From the books (required)

  • Rubin & Chisnell, “Report Findings and Recommendations” (ch 12).

Required

Optional

Extend topic: Website size and other infrastructure issues

9 1 March

Guest lecture: (Tentative - TBD)

Reporting Examples (Brook)

Usability in the field, and moving forward

  • What are the benefits & challenges of conducting usability activities in the field?
  • What are other tools and approaches?

Required

Extend topic: Biometrics

10 8 March

Course wrap up & Presentations

Optional

Acknowledgements

In this version of HCDE517, I extensively draw from and/on materials developed by Rebecca Destello, Andy Davidson, and Judy Ramey.