Real-Time Sound Mapping of Affect: A Wearable-Enabled Biometric Mapping Framework for Emotion Recognition in Music Generation

Abstract

In affective computing and human–computer interaction, interpreting continuous physiological data for real-time creative applications is limited by the trade-off between bulky laboratory hardware and ergonomic commercial wearables. This paper presents a scalable node-level mapping framework refining the biometric data pipeline established by the Strings: Sounds from Human Collective Intelligence project. Grounded in Kurt Lewin’s field theory and James A. Russell’s circumplex model of affect, the system translates peripheral autonomic nervous system signals into continuous musical parameters. A custom wearable prototype collects electrodermal activity and heart rate variability, streaming low-latency data via UDP to Max/MSP and Ableton Live. To resolve interpersonal baselining, we introduce the Zero Point Protocol, a calibration using neutral-valence International Affective Digitized Sounds (IADS-2) stimuli to isolate participant-specific deviations from a resting floor. An affective mapping matrix then updates four-quadrant coordinates to govern tempo, harmony, and timbral modulation. For multi-user contexts, Weighted Harmonic Resolution assigns divergent signals to independent musical layers rather than destructive averaging. Bench testing confirms end-to-end operational stability with low latency; formal validation via Self-Assessment Manikin ratings and Spearman’s rank correlation (𝜌) against IADS-2 benchmarks is outlined as next steps. The framework provides a reproducible model for biometric sound mapping to foster nonverbal social cohesion.



Received: 26 May 2026 | Revised: 15 July 2026 | Accepted: 29 July 2026



Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.



Data Availability Statement

Data are available from the corresponding author upon reasonable request.



Author Contribution Statement

Dario Mattia: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Fabrizio Festa: Conceptualization, Resources, Writing – review & editing, Supervision, Project administration.