KU researcher gets at building blocks of language through speech prosody

LAWRENCE – Even the author admits that one of the main concepts of his new book is difficult to grasp: “Speech prosody, especially intonation, is something that is very elusive. You think you're holding it back, controlling it, and all of a sudden it escapes," said Antônio Roberto Monteiro Simões, University of Kansas professor of Spanish & Portuguese.

This perception that speech prosody is elusive is more easily related to learners of additional languages (or L2, in linguistic jargon), he said.

“I know linguists, native English speakers, who speak French or Spanish very well, who, given their training in linguistics, can understand intonation contours in detail, repeat perfectly what their language teachers tell them to repeat, but when they leave the classroom, their mastery of intonation in the classroom is gone,” Simões said.

That is one reason why Simões wrote a new book, “Spanish and Brazilian Portuguese Pronunciation: The Mainstream Pronunciation of Spanish and Brazilian Portuguese, from Sound Segments to Speech Melodies” (Springer, 2022).

It is Simões’ explanation that speech prosody can be broken down into musical notes and offer new insights on how human languages and other forms of communication function. Prosody is a cover term for everything in speech that goes beyond the sound segment. Vowels and consonants are sound segments, while prosodies include rhythm, sentence intonation, intonation contours at the syllable level, syllable stress, sentence stress, sound quality, phonological rules and other elements that surface with the meaningful linking of segments.   

Simões’ book presents the foundations of a new framework for the analysis of human communication that intends to use a modified version of the Western traditional musical notation to analyze any language. The initial data that he gathered came from Latin American Spanish, Brazilian Portuguese and American English. One of the ideas is that words and sentences can be broken down into musical notes, mostly at the syllable level. Their prosody – perhaps the best single synonym for which is “musicality” – is an intrinsic part of their meaning. It’s the first step in exploring Simões’ unified linguistic theory intended for the analyses of the “melodies” of all human languages.

In Simões' view, musical notation, but not necessarily traditional musical notation, is a powerful and universal means of analyzing human communication. Emotional states, for example, can be represented in musical notes, and human emotions are an area of ongoing research in Simões' work. There are voluntary emotions reflected in speech, in facial and bodily expressions, and in involuntary emotions. In recent years, the study of involuntary emotions has become a new trend in the study of communication. Involuntary emotions like heartbeat and breathing are translatable into rhythmic patterns.

If the rhythmic patterns of involuntary emotions are successfully represented in some kind of musical notation, they will be more easily understood by many people, especially those who read music, and hopefully accessible to all in the future, Simões said.

Simões’ new book is full of charts, graphs, formulas and illustrations that help to explain his theories about sound segments, speech prosodies and speech melodies. Speech melodies are the equivalent term for speech prosodies. Speech prosody deals with spontaneous, daily speech. Speech melodies deal with speech in music, as in opera, for example, when singers talk in a singing-like conversation. In this context, the book illustrations cover such topics as the displacement of air, the duration of sounds, their relative volume, the position of the tongue in the mouth and more.

“It's a very complex production because it deals with music, and I am not a musician,” Simões said, “so I had to learn quite a bit in the last couple of years. And the more I learned, the more I loved the whole project.”

Simões is also not a physicist, but he felt he had to learn some basic physics to write the chapter on “The Acoustics of Speech.”

“For a humanities-trained person, it takes some time to deal with physics,” he said.

Simões goes so far as to plot certain speech prosody on a musical staff. But the work, at its heart, is about language and not music, he said.

“We built all the existing achievements in phonetics and linguistics in general, from a background, from the roots, in music theory,” Simões said. “As I grappled with these ideas throughout my career, I said, 'Why don't we use musical notation to study prosody?' Prosody, and sometimes intonation, are often called the melody of speech. Each language has its particular melody ... its musicality.

“The big problem with the analysis of speech prosody today is that there is no unified framework for analyzing prosody. And generally, the entire existing framework focuses on intonation, not on the other prosodies. So we have major schools of language analysis in England, Holland, France, the United States, for example, and they are all different. These differences make it difficult to talk about the same thing. If we talk about it in terms of the U.S. framework, people here will relate to it. But if you talk about the French framework in the U.S., you may not be able to understand it. ... But if music in the Western world is a universal way of representing speech prosody, well, why not put it that way?”

There are limitations with the use of musical notation, Simões said, but it's all a matter of designing musical notation for speech.

“My idea is different from the historically existing attempts to transcribe speech into musical notation. I think I've gotten past the point where other researchers have tried. My proposal is to change the traditional features of music notation so that you can produce event scores in any type of communication. In the early developments of musical notation we find liturgical representations of speech melodies, i.e., recitative chant as in Gregorian chant, for example, through the use of neumes. Maybe we can call what I do a new neume.

“In other words, traditional music notation must remove some of its fixed divisions appropriate for music but not for speech, language and emotion. Bar lines need to be eliminated so that speech prosody, heartbeats, breathing and other involuntary signs of our emotional states flow in tempo-free and beat-free representations, that is, in a new notation that will allow flexible variations in tempo.”

Image: X-ray with overlay illustrating voice production, from a 1931 book by G. Oscar Russell. Credit: Wellcome Collection