Connecting music and big data

ANN ARBOR—From digital analysis of Bach sonatas to mining data from crowdsourced compositions, researchers at the University of Michigan are using modern big data techniques to transform how we understand, create and interact with music.

Four U-M research teams will receive support for projects that apply data science tools like machine learning and data mining to the study of music theory, performance, social media-based music making, and the connection between words and music. The funding is provided under the Data Science for Music Challenge Initiative through the Michigan Institute for Data Science (MIDAS).

“MIDAS is excited to catalyze innovative, interdisciplinary research at the intersection of data science and music,” said Alfred Hero, co-director of MIDAS and the John H. Holland Distinguished University Professor of Electrical Engineering and Computer Science. “The four proposals selected will apply and demonstrate some of the most powerful state-of-the-art machine learning and data mining methods to empirical music theory, automated musical accompaniment of text and data-driven analysis of music performance.”

Jason Corey, associate dean for graduate studies and research at the School of Music, Theatre & Dance, added: “These new collaborations between our music faculty and engineers, mathematicians and computer scientists will help broaden and deepen our understanding of the complexities of music composition and performance.”

The four projects represent the beginning of MIDAS’ support for the emerging Data Science for Music research. The long-term goal is to build a critical mass of interdisciplinary researchers for sustained development of this research area, which demonstrates the power of data science to transform traditional research disciplines.

Each project will receive $75,000 over a year. The projects are:

Understanding and Mining Patterns of Audience Engagement and Creative Collaboration in Large-Scale Crowdsourced Music Performances

Investigators: Danai Koutra and Walter Lasecki, both assistant professors of computer science and engineering

Summary: The project will develop a platform for crowdsourced music making and performance, and use data mining techniques to discover patterns in audience engagement and participation. The results can be applied to other interactive settings as well, including developing new educational tools.

Understanding How the Brain Processes Music Through the Bach Trio Sonatas
Investigators: Daniel Forger, professor of mathematics and computational medicine and bioinformatics; James Kibbie, professor and chair of organ and university organist

Summary: The project will develop and analyze a library of digitized performances of Bach’s Trio Sonatas, applying novel algorithms to study the music structure from a data science perspective. The team’s analysis will compare different performances to determine features that make performances artistic, as well as the common mistakes performers make. Findings will be integrated into courses both on organ performance and on data science.

The Sound of Text
Investigators: Rada Mihalcea, professor of electrical engineering and computer science; Anıl Çamcı, assistant professor of performing arts technology

Summary: The project will develop a data science framework that will connect language and music, developing tools that can produce musical interpretations of texts based on content and emotion. The resulting tool will be able to translate any text—poetry, prose, or even research papers—into music.

A Computational Study of Patterned Melodic Structures Across Musical Cultures
Investigators: Somangshu Mukherji, assistant professor of music theory; Xuanlong Nguyen, associate professor of statistics

Summary: This project will combine music theory and computational analysis to compare the melodies of music across six cultures—including Indian and Irish songs, as well as Bach and Mozart—to identify commonalities in how music is structured cross-culturally.

The Data Science for Music program is the fifth challenge initiative funded by MIDAS to promote innovation in data science and cross-disciplinary collaboration, while building on existing expertise of U-M researchers. The other four are focused on transportation, health sciences, social sciences and learning analytics.

Hero said the confluence of music and data science was a natural extension.

“The University of Michigan’s combined strengths in data science methodology and music makes us an ideal crucible for discovery and innovation at this intersection,” he said.

More information:

• Data Science for Music Challenge Initiative
• MIDAS