Abstract
There can be no doubt that we often experience correspondences between different sense modalities in music, such as between sound, vision, motion, and touch (just to mention the most prominent ones). This is evident in dance and other kinds of music-related body motion, and also reflected in listeners' innumerable accounts of visual associations with music, and in the ubiquitous use of visual metaphors for musical sound such as "rough", "smooth", "narrow", "broad", etc. In short, it should not be controversial to say that music is a multimodal form of art, that music involves a number of sensations in addition to pure sound. But more precisely how different sense modalities are activated, and how they interact in musical experience, still presents us with a number of unanswered questions. In our research, we have been pursuing the idea of what we call "motormimetic cognition", meaning an incessant mental simulation of sound-related body motion in music perception, primarily of assumed sound-producing body motion (e.g. hitting, stroking, bowing, blowing), but also of various kinds of sound-accompanying body motion (e.g. dancing, walking, gesticulating). We regard such mental simulation of body motion as applicable to most features of music, and we believe motormimetic cognition also can translate between different modalities, for instance that mental simulation of motion can translate between sound and visual images in musical experience. We thus believe motormimetic cognition can be the basis for a systematic research effort on feature mapping between different modalities in music, by way of studying both musicians' and listeners' body motion trajectory shapes and/or posture shapes in music-related contexts. In this chapter, we shall present basic principles of motormimetic cognition and demonstrate how it is relevant for work with new technologies and in multimedia art. Starting out with a review of timescales in our experiences of sound and motion, we shall present a rudimentary taxonomy of sound-motion categories, proceeding in a top-down manner from some global, salient features down to a fairly large number of detail features. This should enable us to have a versatile and flexible conceptual apparatus, as well as a collection of practical tools (using available technologies for sound and motion research), that will allow us to explore our aesthetic and affective images of sound and motion in music, hopefully also contributing to bridging the gaps between quantitative and qualitative approaches in research.
© 2017 Routledge