June 2003

Document Type


Degree Name



Dept. of Computer Science and Engineering


Oregon Health & Science University


In air-traffic control centers, military command posts, and hospital emergency rooms, life-and-death decisions must be made quickly in the face of uncertain information. In these high-stress environments, professionals put a premium on safety, timeliness, team cohesiveness, and mutual awareness. Consequently, these professionals discard computational tools in favor of those that are more robust, malleable, physical, and high in resolution. Indeed, we claim that in order for computational tools to be effective and accepted, they must support these properties. Within this thesis, we argue that it is now possible, using the language of a task, to augment real world artifacts, creating digital tools as robust, malleable, and portable as paper and other physical artifacts. In support of this thesis, we developed Rasa, a tangible augmented reality environment that digitally enhances the existing paper-based command-and control-capability in a military command post. By observing and understanding the users' speech, pen, and touch-based multimodal language, Rasa computationally augments the physical objects on a command post map, linking these items to their digital representations-for example, linking a paper map to the world and Post-it™ notes to military units. Herein, we (1) argue that the properties of physical tools cannot be ignored when designing computational replacements, and demonstrate the effect of doing so in a military command post; (2) identify constraints for the design of fail-safe, mission-critical systems for decision support; (3) present Rasa. a system for augmenting physical objects, transforming them into tangible interfaces; (4) review the related work in augmented reality and paper-based and invisible interfaces; (5) present the findings of an experimental evaluation of Rasa in the field; (6) discuss the results of this evaluation, the limitations of Rasa, its relevance, and the overall impact of Rasa on the design of intelligent systems: (7) expound upon potential avenues of future work, including a vision for Rasa that includes the computational augmentation of arbitrary physical objects.




OGI School of Science and Engineering



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.