51. VIBE MESSAGING: AUGMENTING COMPUTER MEDIATED COMMUNICATION WITH A VIBROTACTILE ENCODING OF SPEECH

Department: Computer Science & Engineering
Faculty Advisor(s): William Griswold

Primary Student
Name: Elizabeth S. Bales
Email: earrowsm@ucsd.edu
Phone: 858-534-9669
Grad Year: 2011

Student Collaborators
Laura Pina, lrpina@cs.ucsd.edu | Kevin Li, k2li@ucsd.edu

Abstract
SMS and instant messaging have become popular forms of Computer-Mediated Communication. They are useful in a number of scenarios where voice calls are not feasible. Unfortunately, because these textual representations of language are unable to include elements of intent and prosody, they can lead to misunderstandings. We propose to address this shortcoming by augmenting the typically text-based Computer-Mediated Communication with a vibrotactile encoding of speech. Enlightened by linguistic properties of speech, we have developed an algorithmic approach to generating a vibrotactile encoding of speech.

In this poster, we present an application called VibeMessaging that allows people to send and receive vibrotactile messages. We make three contributions. First, we present the techniques we employed to encode speech in vibrotactile sequences. Second, we demonstrate how users perceive suprasegmental features of phonetics which help approximate linguistic prosody (rhythm, intonation and vocal stress) to be mapped to vibrotactile sequences. Finally, we represent the results of a formative study, demonstrating how such an application might be used.

We present the results of 3 user studies. Our first user study shows how people map prosody to different forms of vibrotactile language. Our second user study shows how affective aspects of speech can be mapped to the vibrotactile channel, in support of text. These findings guided the development of our VibeMessaging application. Our final user study examines how people use VibeMessaging in the wild.

« Back to Posters or Search Results