Haptics in the Air - Exploring Vibrotactile Feedback for Digital Musical Instruments with Open Air Controllers

Abstract

When playing a traditional musical instrument, the performer often relies on vibration that is produced by the instrument. When perceived through tactile sensing, this can be defined as \emph{vibrotactile feedback}. Since sound in digital musical instruments (DMIs) is not produced by mechanical vibration of its constituent parts, vibrotactile feedback is inherently absent. This means that DMIs are lacking an important feedback modality. DMIs can be played using a wide range of different controllers. \emph{Open air} controllers can make use of motion carried out in open air to control sound. These controllers are particularly prone to the issues related the lack of vibrotactile feedback since they may not have a tangible interface. In this thesis it was investigated how open air controllers can be augmented with vibrotactile feedback. With basis in relevant theory and previous attempts, two DMI prototypes based on open air control of sound were developed. The prototypes allowed control of musical sound on a high and low level. Open air motion was captured using motion capture technology. In this case, the control surface consisted of a tangible element, such that actuators could be embedded in the controller. It was investigated how vibrotactile feedback can convey musical information. This issue was investigated from both a theoretical and practical approach. The practical approach entailed providing vibrotactile feedback to the fingertips of the performer using signals that were synthesized in musical programming environments. Preliminary results of an informal evaluation of the developed vibrotactile feedback strategies suggest that information on musical parameters such as amplitude and timbre can be conveyed with vibrotactile feedback. While the importance of vibrotactile feedback is stressed in the literature, the preliminary results also show that the developed feedback strategies can be found useful.