MUSIC AND LANGUAGE INFO PAGE | research | last updated in 2010
Essential Reference
Guide to Music and Language Cognition
To inform more people, a very concise introduction of music and language cognition research is written here by me in the following sub-sections:
INTRODUCTION Here are general comments and questions about music and language research. Also a good place to start if you want to have a general idea. Here are some of the questions that people have been wondering about in music and language: (1) People who speak with an obvious foreign accent do not necessarily sound like a foreigner in singing (or rather many good singers make you think they're native speakers while they are not). Why? (2) People who stutter in speaking do not stutter in singing. Why? (3) Music can help recovering language functions in brain damage patients. Why? (4) Certain spoken sentences are suddenly heard as singing only because it was recorded and played back a few times. Why? (5) Research shows that the percentage of musicians with absolute pitch is much higher in tone-language speakers than non-tone-language speakers. Why? (6) Adults can begin to learn to pronounce sounds in foreign languages and can understand and be understood. Adults cannot just begin to learn sounds of musical pitch and be able to hear a symphony orchestra in their brains when they see a full score of Beethoven's fifth symphony. Why? (7) Research showed that common people use more of the right part of their brain for music and left part (hemisphere) for language and logic. But composers usually use more of their left hemisphere for music as well. How come? (8) Imagine you're lying there in your bathtub, comfy, and you bury your head under the water for a minute. You hear the people on TV talking. Your ears are under water, and you can only hear some pitch changes, intonations, rhythm, up and down...but you know he is talking in English, not French, not Chinese. But wait a minute, pitch, melody (up and down), rhythm...isn't that components of music that you usually think of? How on earth did your brain pick up "English" from those things? (courtesy of Dr. Carie Benon of Kyushu University, Japan) Some of these questions I used to ask my linguistics prof who didn't know how to answer them. We certainly do not have a definite answer for many of these questions(but at least in my research I finally am coming around to answer some of them that are important to me). In any case, they are definitely a good starting point to think why the study of music and language improves our understanding of how human brain makes sense out of sound, a most unique ability. [Post-introduction comment]: Please also note that comparing music and language builds upon a basic assumption that they share cognitive abilities that are unique and not shared by other cognitive tasks. However, that is questionable, as appealing as it sounds. Read Ray Jackendoff's recent article on how we should be more careful when generalizing conclusions about the comparison. Also, recent advancements in language sciences also call into question the unique cognitive function of language--how language evolution shared some of the cognitive abilities found in more general cognitive skills--read it here (BBC and Nature--[full text on Nature here]).As the endeavor of comparing music and language thrives, I believe paying increasing attention to evidence in this respect is an inevitable way to go. Please refer to the top of the page for individual subsection links to learn more about the research on music and language. SOUND ELEMENTS
Music and language are both complex sound systems unique to human beings. Music utilizes pitch as the main device of contrast. Language, on the other hand, depends primarily on timbre contrast (imagine speaking in a monotone and still being intelligible), although both pitch and timbre do play an important role in both music and language. Music and language both involve sound categories, inventory of sound elements particular to a cultural musical/linguistic system. Children are born to possess the ability to learn any language of the world (or any music for that matter), yet their ears become more and more selective while adapting to the sounds in the cultural system that they grew up with, and eventually they lose the ability to naturally acquire unfamiliar languages. Do music and language sounds share common neurological processing? The general answer is that while they may share some common processing resources, music and language sounds very likely have different representations in the brain (specialization). An interesting line of study is to investigate how the process of specialization is executed in the childhood regarding musical sound, linguistic sound, and other non-speech sound. In addition, specialization in one area can also affect the perception on other categories. Pitch perception in music usually involves a distinction between relative and absolute pitch judgment. Few professional musicians in the West possess absolute pitch. More in East Asia. Recent research shows that this might be relevant to the pitch categorization and linguistic ability of absolute pitch in the tone languages. RHYTHM
Linguistic rhythm includes parameters like duration and stress (and?). Earlier notions (based on our subjective perception) divides world languages into several categories in terms of rhythmic typology: stress-timed language, syllable-timed language, mora-timed language, etc. The hypothesis of isochrony depends on the periodicity of the linguistic rhythm: equal duration between stress for stress-timed languages (e.g.English), equal duration between syllables for syllable-timed languages (e.g. French). The distinction in rhythm such as those in English and French is indeed useful when, for instance, serving as a guideline to learn a second language with a different rhythmic typology, although failed empirical tests later on. The new rhythmic typology is based on the notion of nPVI*, pointing to the duration contrast, degree of vowel reduction, diversity of syllable structure, etc. of the target language. It was advanced by Patel and other scholars that the nPVI shows the influence of linguistic rhythm on musical rhythm for groups of representative composers found representative in a nationality. Patel proposed that although the notion of isochrony failed empirical test, we should nevertheless broaden our concept of rhythm to any pattern that occur in time (follow certain sequential rules) without having to satisfy the strict notion of equal-length periodicity. However, he also paid attention to the unique ability of human to synchronize to any musical beat without much effort while no known animal has been documented to have such ability. Later discovery of a bird (a cockatoo named snowball) who can perfectly synchronize to a musical beat added to the myth of this matter. (snowball dancing to a musical beat on Youtube) In the process of learning and memory, research also showed a strong effect of adding rhythm to verbal learning among children of a young age. In ancient China, texts are written in sentences with equal syllable length, which was thought to greatly facilitate to the recitation by heart of a great amount of classic essays. Recent textbooks on utilizing this effect is also seen in the field of teaching Chinese as a second language. MEANING
Recent research in cognitive musicology identified 11 types of musical meaning, ranging from intrinsic (e.g., musical structure), extrinsic (e.g., the expression and experience of emotion), to meta-musical meaning on a societal level (e.g., social associations; musicalstructure and cultural concepts) (Patel 2008-MLB). Previous studies on Western art music identify two categories of cues that are important to the perception of musical expression in association with certain types of emotions: (1) those that correspond more universally with human psychology/physiology, e.g., tempo, pitch register, timbre; (2) those that are more cultural specific, e.g., majorand minor keys in Western art music (Balkwill & Thompson 1999:48). Music and Emotion Bibliography (Dr.David Huron, Ohio State University) SYNTAX
Syntax is the grammatical construction/organization of a logical system (such as music and language). Generative Grammar is one of the most widely known theories of linguistics since the time of Chomsky's advancement. Conductor and composer Leonard Bernstein also got inspired, leading to the Six Harvard Lectures entitled Unanswered Questions and a book of the same title. Although Bernstein's theory was subjective, it did lead to a series of events which eventually led to the publication of the Generative Theory for Tonal Music by Lerdahl and Jackendoff. Although this theory is inspired by linguistic theory, it did not resemble linguistic syntax due to the differences of music and language. In fact, until recently, Jackendoff always calls for cautions not to draw close comparisons between the two unless based on solid evidence (see Parallels and Non-Parallels between Language and Music, Music Perception 26, 195-204.) In 2009, Pesetsky and Kates from MIT re-organized the generative theory of music by a series of formal deductions, and argued that the only difference on music and language is their building blocks. In other aspects they are identical. Recent advancement in neuroscience has enabled scientists to study the syntactic processing of music and language by looking at the brain imaging data. Patel found an interference effect of musical and linguistic syntactic processing by using groups of syntactically normal and tricky (garden path sentences and abnormal harmonic progression) signals. Semantic interference served as a control group and the interference was not observed. For a comprehensive review of the modularity of brain in speech and singing see Peretz 2009. EVOLUTION
The whole argument of music and language probably started from a simple question regarding evolution: did people start singing or speaking first? It seems that there are hard evidence that people started talking in a pretty late time in the history of homosapien, at least much later after they walked out of Africa and scattered throughout the world. Up until then people did need to communicate as primate living a group-based society. One hypothesis is to use pitch rise and falls without any articulation. But how elaborate are these pitch/melody based communications? Recent studies by Steven Brown (The Origin of Music), Ian Cross, and Joseph Jordania (Who Asked the First Question: Origins of Vocal Polyphony, Human Intelligence, Language and Speech) may have something to say about that. Anthropologists found that throughout the world, down to many secluded tribes deep in the Amazonian forest, people can live a much simpler life than we assume, without many elements of our culture, such as myth, legend, visual art, the ability to count, or the fixed terms for color. But all of them have music and language. "Music and language define us as human", a statement indeed true. On the one hand, the evolutionary value of language is pretty obvious. On the other hand, the value of music in evolution is not clear. According to Patel, there are no solid evidence so far pointing to the evolutionary advantage of music. Why music, sometimes considered as purely a byproduct, an optional entertainment, something for pastime, appeared to be non-optional when comes to define the human race? One hypothesis points to the ancient origin of music. Music evolved into something so powerful and so persistent long before speech so it tends to stick on. Another explanation proposed by Patel is that, music is a "transformative technology" that once created, human could not and do not want to live with out it. It is kind of like the ability of making fire. It went on as a unbreakable tradition and also modifies our brain constantly. Music can also stimulate the identical response in our brain as food and sex do (the 'rewarding' center of the brain). UPDATE:The recent study of the communication among non-human animals, particularly our close relatives, such as bonobos, champs, etc., revealed much fundamental similarity with human language, including in-species recognized dialects and combination of syntactic components into meaningful sequences of signals.Some of these studies, to me, also reveals the truth about the origin of language and music--it is very likely that something in between was used as a means of communication in early human history, not unlike what we observed from other species today. Evolution:Selected Bibliography Ian Cross-Music and Evolution Publications Ani Patel-Publications (with many on evolution) |
Presentation at CASS: MUSIC and LANGUAGE | research
[Jan.6, Thu, 1:30pm, 2010/6th floor conference room (641), CASS/中國社會科學院語言研究所語音實驗室]
SHUO ZHANG/THANK YOU FOR ATTENDING!
PART I Music and Language: Overview
The relationship between music and language has attracted scholarly debate over a long period of time in history, from various perspectives.Recent years have witnessed a growing interest in the scientific study of language and music by cognitive and neural scientists, linguists, among others.The goal of such endeavors usually involves a central question that yet to be resolved, which is, “what perceptual and cognitive operations are specific to language, and which are also involved when processing other types of well-constructed and organized signals, such as music?” However, recent research from interdisciplinary perspectives showed that both music and language are complicated cognitive processes that require bi-hemispheric processing from multiple parts of the brain; therefore it is often favorable to limit the research scope to specific areas and levels instead of talking about music and language as single entities. Thus a more general shift in the research scope emerges: (Patel 2008:417): “[No matter similar or different,] Comparing music and language provides a powerful way to study the mechanisms that the mind uses to make sense out of sound.” This is especially true when we consider that music and language both involve multiple levels of representation. In this first part of the presentation I am going to address the research literature in the specific four areas: pitch/melody, rhythm, syntax, and evolution.
PART II Speech-to-song Illusion: Evidence from Mandarin Chinese
In this part I talk briefly about one of my most recent (and current) research, speech-to-song illusion, which basically addresses the recently revealed perceptual transformation from speech to song-what will happen if we simply repeat some sentences over and over again. Speech is transformed into song only by repeating the sound signal, while constraints on the acoustic properties of the signal is investigated. The role of target stability vs. the potential of song-to-speech transformation. This perceptual illusion is first described by Deutsch (2008) in English language and was tested for German language by Falk and Rathecke (2010).
My project seeks to understand the mechanisms of speech to song illusion from a variety of perspective. In this paper I will (1) offer a re-analysis of the data on speech to song illusion by Deutsch (2008) by giving consideration to the research in speech synthesis; (2) test the possibility (and if possible, mechanism) to create speech-to-song illusions in Mandarin Chinese, by applying the target stability hypothesis (Falk and Rathecke 2010) and Target Approximation Model (Xu 2002); (3) test the Falk-Rathcke (2010) hypothesis on the interval structure in the speech-to-song illusion in Mandarin Chinese.
Preliminary results were presented at Sino-European Winter School of Logic, Language, and Computation (SELLC2010). Detailed perceptual experiments are yet to be carried out.
PART III Research Prospects
Some of my possible prospective research projects at the institute here are described briefly in this section.
SPEECH SYNTHESIS, INDIVIDUAL SPECTRAL DIFFERENCE, AND THE PERCEPTION OF ENGLISH ACCENT-when the recording of a non-native English speaker is re-synthesized into a different set of voice quality, it is perceived much more like a native speaker.
THE PITCH CONTOUR OF THE INDIAN ENGLISH AND AMERICAN ENGLISH-Indian English has a different pitch stability and contour as measured in syllable unit than American English, which is largely responsible for the creation of their accent. Transformation of the two variations is shown in Prosogram software (Praat scripting). Possible extension to English speakers from China.
SPEECH TO SONG ILLUSION: EVIDENCE FROM MC-speech is transformed into song only by repeating the sound signal, while constraints on the acoustic properties of the signal is investigated. The role of target stability vs. the potential of song-to-speech transformation.
BRAIN SPECIALIZATION OF MUSICAL, LINGUISTIC, AND NON-SPEECH SOUNDS-Previous research showed that (1) there may be evidence pointing to the noted differences of pitch processing between tone languages(e.g.Mandarin) and non-tone languages(English), and in turn pointing to the strong links between musical pitch processing and linguistic pitch processing (absolute pitch) (Deutsch 2006). (2)Evidences pointing to the influence of linguistic pitch processing to non-speech sound processing. (3) also brain specialization (brain modularity) of these different sounds. Investigation of the interactions and specializations of these processes in the childhood from a developmental perspective is called for.
MAPPING OF TONE INTERVALS ONTO TONE CATEGORIES(MC)-my research indicates that realization of tone intervals is robust in third to first tone transition, and significant inconsistency from the canonical forms is observed in real speech.
REFERENCE
Deutsch, D., Lapidis, R., Henthorn, T., 2008. The speech-to-song-illusion. J. Acoust. Soc. America, s.124, 2471.
Deutsch, D., Henthorn, T., Marvin, E., & Xu H-S. Absolute pitch among American and Chinese conservatory students: Prevalence differences, and evidence for a speech-related critical period. Journal of the Acoustical Society of America, 2006, 119, 719-722. [PDF Document]
Falk, F., Rathcke, T. 2010. “On the Speech-To-Song Illusion: Evidence from German”. Speech Prosody 5th International Conference, Chicago, IL.
Patel, A., 2008. Music, language and the brain. Oxford: University Press.
Xu, Y. and Sun X. 2002. Maximum speed of pitch change and how it may relate to speech. Journal of the Acoustical Society of America 111: 1399-1413.
-------------------------------------------
Post-discussion Memo on Speech to song illusion
1. the F0 range of speech versus singing; usually in speech it is very small.
2. the pitch register and pitch spacing of speech and singing. When register is high, how is the same interval mapped onto the frequency ratio and difference(absolute difference)?
3. singing is taking the prosody part (which in language often responsible for emotion) and to exaggerate it.Speech sacrifices acoustic details in favor of the speed of pitch change and
PART I Music and Language: Overview
The relationship between music and language has attracted scholarly debate over a long period of time in history, from various perspectives.Recent years have witnessed a growing interest in the scientific study of language and music by cognitive and neural scientists, linguists, among others.The goal of such endeavors usually involves a central question that yet to be resolved, which is, “what perceptual and cognitive operations are specific to language, and which are also involved when processing other types of well-constructed and organized signals, such as music?” However, recent research from interdisciplinary perspectives showed that both music and language are complicated cognitive processes that require bi-hemispheric processing from multiple parts of the brain; therefore it is often favorable to limit the research scope to specific areas and levels instead of talking about music and language as single entities. Thus a more general shift in the research scope emerges: (Patel 2008:417): “[No matter similar or different,] Comparing music and language provides a powerful way to study the mechanisms that the mind uses to make sense out of sound.” This is especially true when we consider that music and language both involve multiple levels of representation. In this first part of the presentation I am going to address the research literature in the specific four areas: pitch/melody, rhythm, syntax, and evolution.
PART II Speech-to-song Illusion: Evidence from Mandarin Chinese
In this part I talk briefly about one of my most recent (and current) research, speech-to-song illusion, which basically addresses the recently revealed perceptual transformation from speech to song-what will happen if we simply repeat some sentences over and over again. Speech is transformed into song only by repeating the sound signal, while constraints on the acoustic properties of the signal is investigated. The role of target stability vs. the potential of song-to-speech transformation. This perceptual illusion is first described by Deutsch (2008) in English language and was tested for German language by Falk and Rathecke (2010).
My project seeks to understand the mechanisms of speech to song illusion from a variety of perspective. In this paper I will (1) offer a re-analysis of the data on speech to song illusion by Deutsch (2008) by giving consideration to the research in speech synthesis; (2) test the possibility (and if possible, mechanism) to create speech-to-song illusions in Mandarin Chinese, by applying the target stability hypothesis (Falk and Rathecke 2010) and Target Approximation Model (Xu 2002); (3) test the Falk-Rathcke (2010) hypothesis on the interval structure in the speech-to-song illusion in Mandarin Chinese.
Preliminary results were presented at Sino-European Winter School of Logic, Language, and Computation (SELLC2010). Detailed perceptual experiments are yet to be carried out.
PART III Research Prospects
Some of my possible prospective research projects at the institute here are described briefly in this section.
SPEECH SYNTHESIS, INDIVIDUAL SPECTRAL DIFFERENCE, AND THE PERCEPTION OF ENGLISH ACCENT-when the recording of a non-native English speaker is re-synthesized into a different set of voice quality, it is perceived much more like a native speaker.
THE PITCH CONTOUR OF THE INDIAN ENGLISH AND AMERICAN ENGLISH-Indian English has a different pitch stability and contour as measured in syllable unit than American English, which is largely responsible for the creation of their accent. Transformation of the two variations is shown in Prosogram software (Praat scripting). Possible extension to English speakers from China.
SPEECH TO SONG ILLUSION: EVIDENCE FROM MC-speech is transformed into song only by repeating the sound signal, while constraints on the acoustic properties of the signal is investigated. The role of target stability vs. the potential of song-to-speech transformation.
BRAIN SPECIALIZATION OF MUSICAL, LINGUISTIC, AND NON-SPEECH SOUNDS-Previous research showed that (1) there may be evidence pointing to the noted differences of pitch processing between tone languages(e.g.Mandarin) and non-tone languages(English), and in turn pointing to the strong links between musical pitch processing and linguistic pitch processing (absolute pitch) (Deutsch 2006). (2)Evidences pointing to the influence of linguistic pitch processing to non-speech sound processing. (3) also brain specialization (brain modularity) of these different sounds. Investigation of the interactions and specializations of these processes in the childhood from a developmental perspective is called for.
MAPPING OF TONE INTERVALS ONTO TONE CATEGORIES(MC)-my research indicates that realization of tone intervals is robust in third to first tone transition, and significant inconsistency from the canonical forms is observed in real speech.
REFERENCE
Deutsch, D., Lapidis, R., Henthorn, T., 2008. The speech-to-song-illusion. J. Acoust. Soc. America, s.124, 2471.
Deutsch, D., Henthorn, T., Marvin, E., & Xu H-S. Absolute pitch among American and Chinese conservatory students: Prevalence differences, and evidence for a speech-related critical period. Journal of the Acoustical Society of America, 2006, 119, 719-722. [PDF Document]
Falk, F., Rathcke, T. 2010. “On the Speech-To-Song Illusion: Evidence from German”. Speech Prosody 5th International Conference, Chicago, IL.
Patel, A., 2008. Music, language and the brain. Oxford: University Press.
Xu, Y. and Sun X. 2002. Maximum speed of pitch change and how it may relate to speech. Journal of the Acoustical Society of America 111: 1399-1413.
-------------------------------------------
Post-discussion Memo on Speech to song illusion
1. the F0 range of speech versus singing; usually in speech it is very small.
2. the pitch register and pitch spacing of speech and singing. When register is high, how is the same interval mapped onto the frequency ratio and difference(absolute difference)?
3. singing is taking the prosody part (which in language often responsible for emotion) and to exaggerate it.Speech sacrifices acoustic details in favor of the speed of pitch change and