Resources For Research

Designing & conducting experiments

Designing and running human factors experiments: Bjoern recommends Jake Wobbrock’s (UW) and Scott Klemmer’s (UCSD) MOOC on this topic, as well as the book Measuring the User Experience (free electronic resource on OskiCat; print version available) Statistical hypothesis testing: Wikipedia has a pretty good overview of analyzing interventions to see if a tool or methodology does or does not provide some particular benefit to the student. Also see Koji Yatani’s course pages on “Stats for HCI”, including a cheat sheet for “which statistical test should I use for…” For evaluating tools specifically, a few valuable resources:

• The System Usability Scale: a “quick and dirty” reliable tool for measuring tool usability
• NASA TLX (task load index): a widely used, subjective, multidimensional assessment tool that rates perceived workload in order to assess a task, system, or team’s effectiveness or other aspects of performance
• A practical guide to controlled experiments of software engineering tools with human participants, by Amy J. Ko et al. Aimed primarily at tools supporting software engineering tasks and processes.

 

Hints on preparing IRB protocols

Remember that you must complete CITI training and, in all likelihood, file an IRB protocol for any study in which students are involved as subjects and whose data you plan to use in a publication. TBD: hints

Books

• How Learning Works (pdf)
• Handbook of Research on Learning and Instruction (google search, first hit)

Other sites for Education Papers and Courses

• Kristin’s Annotated Bibliography from SME210, the MACSME first year paper review seminar
• MITx working papers
• HarvardX working papers
• Lytics Lab Publications (Stanford)
• MOOC Research Hub Reports (Bill & Melinda Gates Foundation)