Multimodal Perception

Stefania Serafin just held a great lecture on multimodal perception and sonic interaction design at the SMC summer school. It is fascinating with the differences in spatiotemporal resolution between the different senses: Spatial resolution eye — highest: very high spatial acuity (foveal acuity ≈ 1 arcmin), excellent for fine detail. tactile — medium: fingertip resolution ≈ 1–2 mm, good for textures and small features. ear — lowest: coarse spatial localization (degree-level accuracy), poor for fine spatial detail. Temporal resolution ear — highest: excellent temporal sensitivity (sub-ms to ms scale), great for timing and rapid changes. tactile — medium: temporal resolution on the order of a few ms to tens of ms. eye — lowest: relatively slow temporal processing (temporal integration ~50–200 ms; critical flicker fusion ≈ 50–60 Hz). Stefania showed a number of examples of the application of this knowledge in interaction design, one example being the tactile floor they are building at McGill.