User Interface Testing and Evaluation

Factors affecting choice of evaluation methodology

• Stage of design
• Novelty of product
• Expected number of users
• Criticality of interface
• Cost of product and budget for testing
• Experience of design and evaluation teams

Problems with testing

• Can't guarantee perfection
• When do stop repairing bugs and ship the prototype?
• Testing focuses on normal product use, not extreme

Review Methodologies

Expert Review Formats

• Heuristic evaluation
• Guideline review
• Consistency inspection
• Cognitive walkthrough
• Formal usability inspection

Usability Laboratory Approaches

• Think aloud
• Video tape
• Field tests
• Destructive testing
• Competitive usability testing

Surveys

• Focus on specific UI features
• Plan statistical analysis before data collection
• Consider on-line surveys to increase response rates
• Use semantically anchored Likert type scales

Acceptance Tests

• Must be based on measurable criteria
• Focus on user behavior and task completion

Evaluation During Active Use

• Interviews
• Continuous user performance data logging
• On-line or telephone consulting
• On-line suggestion box or trouble reports
• On-line bulleting board or newsgroup
• User newsletter or conferences

Classical Experimental Psychology Methodology