back to syllabus

HCDE517: Usability Analysis - Schedule - W26

Week	Dates	Topic	Readings	Assignments due
1	6 January	Course overview Introductions What is usability? History of usability? Learning & Memory	Getting oriented C Goldberg. “Design Flaws In Electronic Health Records Can Harm Patients, Study Finds,” WBUR. WC Tomlin. UX Killed Usability. Overview from the books Required: Rubin & Chisnell, Chapters 1-3 Hertzum, Chapters 1 & 2 Optional: Tullis & Albert, “Introduction”	Read project descriptions & respond to project survey. Sign up for reading presentation
2	13 January	Planning & considerations for usability testing Why do we need usability testing? Usability testing within the UCD process Identifying usability concerns Usability inspection methods (heuristic evaluations, cognitive walkthroughs) Workshop: Identifying & developing usability concerns and questions	Make sure you have read intros from last week! Usability evaluations - Required J Sauro, What's the Difference between A Heuristic Evaluation and a Cognitive Walkthrough? S Rosenbaum, Usability Evaluations Versus Usability Testing: When and Why? J Nielsen, “Usability Heuristics” (ch5), Usability Engineering (Amazon, UW Libraries). Guidelines for usability - Optional JD Gould & C Lewis. 1985. Designing for Usability: Key Principles and What Designers Think. Communications of the ACM 28(3). Research-Based Web Design & Usability Guidelines, 2024, from Digital.gov. Heuristic evals for specific settings (extend topic, optional) M de Jong & T van der Geest, Characterizing Web Heuristics. J Mankoff, AK Dey, G Hsieh, J Kientz, S Lederer, & M Ames. 2003. Heuristic evaluation of ambient displays. (UW Library Proxy). CHI 2003. E Bertini, S Gabrielli, & S Kimani. 2006. Appropriating and assessing heuristics for mobile computing (UW Library Proxy). AVI 2006. R Langevin, R Lordon, T Avrahami, B Cowan, T Hirsch, & G Hsieh. Heuristic Evaluation of Conversational Agents. (UW Library Proxy). CHI 2021.	Connect with your project group & client.
3	20 January	Planning for a usability study, continued Planning a study that works with your constraints Role of interviews and focus groups in usability studies Selecting representative users Age, Accessibility, and usability	Planning for Usability Testing Required Rubin & Chisnell, Develop the Test Plan (ch 5), Set Up a Testing Environment (ch 6), Find and Select Participants (ch 7) Hertzum, Designing and Planning The test (become familiar with domain and prototype, recruit users, how man users are needed, ch 5.1-5.4) Tullis & Albert, Background (ch 2) Optional S Krug, Usability Testing on 10 cents a day (ch 9) from Don't Make Me Think: A Common Sense Approach to Web Usability. New Riders 2000. J Nielsen. (2007). Fast, Cheap, and Good: Yes, You Can Have it All. Number of participants This is covered in the books; readings below are an optional deeper dive. J Nielsen. 2000. Why You Only Need to Test with 5 Users. J Sauro. 2010. Why You Only Need To Test With Five Users (Explained). L Faulkner. 2003. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. J Dumas, How Many Participants in a Usability Test is Enough? M Schmettow. Sample size in usability studies. Communications of the ACM. 2012. Accessibility (extend topic, optional) W Quesenbery. Accessibility First—for a Better User Experience for All, UX Matters. What are the differences between universal design, accessibility, and inclusive design? SayYeah. Web Accessibility in Mind Petrie & Bevan. The evaluation of accessibility, usability, and user experience in The Universal Access Handbook. Usability, children, and aging (extend topic, optional) RL Franz, C Munteanu, BB Neves, R Baecker. Time to Retire Old Methodologies? Reflecting on Conducting Usability Evaluations with Older Adults. MobileHCI '15. (UW Library Proxy) J Nielsen. Seniors As Web Users, NN Group A Kendrick. Usability Testing with Minors: 16 Tips, NN Group. E Rose, E Bjorling, A Kim, NY Alvarez. Usability testing with teens: Adapting human-centered design and UX methods, SIGDOC 2018.	P1. Preliminary proposal
4	28 January	Forming an initial test plan What are the responsibilities of a tester? What are general considerations to drive a test plan? What is essential and what is optional when preparing a meaningful study? How many methods should you use? Alternative methods? Ethics and usability Usability and automation	Usability test plans & roles Required Rubin & Chisnell, Skills for Test Moderators (ch 4) Tullis & Albert, Planning a Usability Study (ch 3) Optional S Dray & D Siegel. “Penny-Wise, Pound-Wise: Making Smart Trade-offs in Planning Usability Studies.” J Sauro. “What goes into a good usability test plan? Other methods, considerations about how many? MC Medlock, D Wixon, M McGee, & D Welsh. The Rapid Iterative Test and Evaluation Method (RITE): Better Products in Less Time. Walji et al. Are Three Methods Better Than One? A Comparative Assessment of Usability Evaluation Methods in an EHR?, IJMI 2014. Ethics Required: Code of Conduct of the User Experience Professionals Association (2013) Optional A Yadlin-Segal, R Tsuria, W Bellar. Ethics of Studying Digital Contexts. 2020. B Ray. User Research Ethics: Resources. UXPlanet. Usability and automation (extend topic, optional) 99% Invisible: Children of the Magenta (Automation Paradox, pt. 1) 99% Invisible: Johnnycab (Automation Paradox, pt. 2) S Nadeem. The Deadly Price of the Automation Paradox. The Walrus.	P2. Interaction map Draft: in class Final: Sunday, 1 February
5	3 February	Preparing a test with measurable results What test materials do we need for a usability study? What will be measured? How do we create successful data collection instruments? Workshop: Study plans	Preparing test materials (required) Rubin & Chisnell, Prepare Test Materials (ch 8). Hertzum, Make tasks (ch 5.5) Metrics and measures Required Tullis & Albert, Performance Metrics (ch 4), Issues-based Metrics (ch 5), Self-reported Metrics (ch 6), Behavioral and Physiological Metrics (ch 7) J Dumas, Usability Testing Methods: Subjective Measures, in The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications, 3^rd ed. Optional J Sauro & J Dumas. Comparison of Three One-Question, Post-Task Usability Questionnaires, CHI 2009. (UW Library Proxy) H Suh, N Shahriaree, EB Hekler, JA Kientz. Developing and Validating the User Burden Scale: A Tool for Assessing User Burden in Computing Systems. CHI 2016. Remote testing (extend topic, optional) B Völkers. A beginner’s guide to remote usability testing MS Andreasen, HV Nielsen, SO Schroder, J Stage. 2007. What happened to remote usability testing?: an empirical study of three methods. (UW Library Proxy.) CHI 2007. Remote Moderated Usability Testing. Nielsen Norman Group. Moderated and Unmoderated Think Aloud Comparison. (2015) Usability of Services (extend topic, optional) SA Munson, EC Friedman, K Osterhage, R Allred, MD Pullmann, PA Areán, AR Lyon. 2022. Usability Issues in Evidence-Based Psychosocial Interventions and Implementation Strategies: Cross-project Analysis, JMIR 24(6). A Lyon et al. The Cognitive Walkthrough for Implementation Strategies (CWIS): a pragmatic method for assessing implementation strategy usability. B Carson. Uber is making changes to eliminate the fear factor for drivers, Business Insider.	P3. Study plan Draft in class Final by Sunday, 8 Feb
6	10 February	Conducting a usability study What is “thinking aloud”? What is key to successful facilitation? How do we collect data from a usability study? Workshop: Study kits	Conducting test sessions Required Rubin & Chisnell, Conduct the Test Sessions (ch 9). Hertzum, Execution: Running the Test Sessions (ch 6). Optional M Hawley. “User Interview Techniques, Guidelines for Obtaining Better Results,” User Experience 8(3). M Margolis. “How to hack your body language for better interviews,” Design Staff. Think Aloud Testing Required: J Ramey. Methods for successful “Thinking Out Loud” procedures. Optional T Boren & J Ramey. (2000). “Thinking Aloud: Reconciling Theory and Practice,” IEEE Transactions on Professional Communication 43(3). M Norgaard & K Hornbaek, “What Do Usability Evaluators Do in Practice? An Explorative Study of Think-Aloud Testing,” (UW Library Proxy). DIS 2006. Invisible OSs, Recognition, and Recall (extend topic, optional) R Budiu. Memory Recognition and Recall in User Interfaces Nielsen Norman Group. J Pavlus. "No UI" Design's Next Move: Fake UI. FastCompany D Norman & B Tognazzini. How Apple Is Giving Design A Bad Name. FastCompany Deception in UI/UX (extend topic, optional) Press me! The buttons that lie to you BBC Future E Adar & J Teevan. Benevolent Deception in Human Computer Interaction, (UW Library Proxy), CHI 2013. Blank Page Technique and Paper Prototyping (extend topic, optional) Still & Morris. 2010. The Blank-Page Technique: Reinvigorating Paper Prototyping in Usability Testing, IEEE Transactions on Professional Communication 53(2). (UW Library Proxy) Paper Prototyping is a Waste of Time The Magic of Paper Prototyping	P4. Study kit Draft: In class Final: by Sunday, 15 Feb
7	17 February	Data analysis & reporting Validity and Reliability Analysis and Reporting Workshop: Data analysis	Analyzing Data Required Rubin & Chisnell, Analyze Data and Observations (ch 11). Hertzum, Analyzing the Data, How Many Evaluators are needed?, Rate problem severity (ch 7.1-7.3). Tullis & Albert, Combined and Comparative Metrics (ch 8). Optional C Wilson. Analyzing and Reporting Usability Data, STC Usability SIG Newsletter. M Hughes. Rigor in Usability Testing, Technical Communication, Fourth Quarter (1999): 488-94. J Sauro & J Lewis. Correlations Among Prototypical Usability Metrics: Evidence for the Construct of Usability, (UW Library Proxy). CHI 2009 L Kanter, D Sova, & L Anschuetz. Organizing Qualitative Data from Lab and Field: Challenges and Methods, UPA 2005. Voice User Interfaces & Usability (extend topic, optional) Voice Interaction UX: Brave New World...Same Old Story. Nielsen Norman Group. Is Voice UI the future of user experience design? UX Magazine A/B testing in UX/usability (extend topic, optional) A/B Testing Log data to understand UX and address usability issues (extend topic, optional) S Farrell. Search-Log Analysis: The Most Overlooked Opportunity in Web UX Research, NN Group. MC Burton & JB Walther. The Value of Web Log Data in Use-Based Design and Testing. Journal of Computer Mediated Communication 2006. A D'Costa. Tracking User Behaviors on Web Sites: Why It Matters and How to Do It, UX Matters.
8	27 February	Presenting your findings Discuss details for the final assignments How can data and findings be meaningfully shaped for stakeholders? Reporting and communicating results	Reporting Results Required Rubin & Chisnell, Report Findings and Recommendations (ch 12). Hertzum, Devise Redesign Proposals (ch 7.4), Report Test Findings (ch 7.5). A Schade. 2013. Making Usability Findings Actionable: 5 Tips for Writing Better Reports. Optional D Yeats & L Carter. The Role of the Highlights Video in Usability Testing: Rhetorical and Generic Expectations, Technical Communication, May 2005: pp. 156-162(7). J Nielsen. 2005. Formal Usability Reports vs. Quick Findings. CE Wilson & Kara P. Coyne. 2001. Tracking Usability Issues: To Bug or Not to Bug?, ACM interactions. Additional Approaches (extend topic, optional) K Rodden, H Hutchinson, & X Fu. Measuring the User Experience on a Large Scale, CHI 2010. (UW Library Proxy). S Ehrlich, K Cross, W Chan, D Imm. More for Less: A Novel Hybrid Method to Maximize the Impact of Research, Ext. Abstracts CHI 2003. Technical choices that affect usability (extend topic, optional) T Biddle. Site Speed and Usability
9	3 March	Usability in the field, and moving forward What are the benefits & challenges of conducting usability activities in the field? What are other tools and approaches? Panel Reporting (continued)	S Reeves. How UX Practitioners Produce Findings in Usability Testing. ACM Transactions on Computer-Human Interaction 26(1): article 3. Critiques (required) E Friess. 2010. The Sword of Data: Does Human-Centered Design Fulfill its Rhetorical Responsibility? Design Issues 26(3). Greenberg & Buxton. Usability Evaluation Considered Harmful (Some of the Time), CHI 2008. P Dourish. 2019. User Experience as Legitimacy Trap. interactions 26(6):46-49. Biometrics (extend topic - optional) J Kahn. Expedia Reads Your Mind (and Face) to Beat Rivals.
10	10 March	Course wrap up & Presentations	Various takes on the state of usability over time (optional) J Redish. 1995. Are we really entering a post-usability era? SIGDOC Asterisk J. Comput. Doc. 19(1): 18-24. J Kelway. Usability is dead. N Tractinsky, AS Katz, D Ikar. 2000. What is beautiful is usable, Interacting with Computers 13(2): 127-145.	P5a. Presentation P5b. Report Self & Team Evaluations

back to syllabus