Andi Lushaj


May 2010

Document Type


Degree Name



Dept. of Medical Informatics and Clinical Epidemiology


Oregon Health & Science University


BACKGROUND: Computer-mediated educational applications can provide a self-paced, interactive environment to deliver educational content to individuals about their health condition, resulting in improved overall health and reduced health care costs. OBJECTIVE: There is a need to adequately evaluate health information systems to show evidence that justifies investment in such systems. We evaluated the feasibility of remote usability testing of the health coaching software, as an easier to arrange and cheaper method of testing users in their own environment and more frequently. METHODS: One evaluator carried out a heuristic evaluation of the Automated Health Coaching System interface based on Nielsen’s heuristics, and generated a list of usability problems with severity ratings. For the usability testing, five participants used the interface and scenarios to complete 11 tasks. We measured the elapsed testing time on task, the number of errors made per task, and the task accuracy. Sessions were recorded to capture the participant’s screen and mouse movement, sound, and webcam for later analysis. RESULTS: The remote usability evaluation found a total of 146 usability problems. The heuristic evaluation violations mostly consisted of lack of consistency and standards, lack of an aesthetic and minimalist design, failure of the system to match the real world, and lack of visibility of system status. The usability testing showed problems in areas like viewing and interpreting the performance graphics, editing of issues, locating a subject’s active topic or contacts, or making the correct selections within contacts and issues. CONCLUSION: The findings of this study demonstrate that the heuristic evaluation and remote synchronous usability testing are efficient methods for interface evaluation and design improvement.




School of Medicine



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.