This site is only a doorway

Everything here is gathered from things other people actually researched, thought through, and built: books, papers, historical records, and the people behind them. If something here catches your interest, go look at the original sources listed at the bottom of the page. They carry more depth than a page like this ever can.

Can Your Brain Fill in Missing Sounds and Read Emotions from a Voice?

Category: The Body

In one experiment, researchers replaced a single sound in a word with a cough. Out of 20 people, 19 did not notice the missing sound. That is 95 percent of the participants [1]. The noise reaching the ear was different, yet people felt they heard the complete word. How does this happen?

This article explores how the brain fills in missing sounds, how it chooses which voice to listen to in a noisy room, and how it reads emotions like smiles from a voice. We look at what experiments have shown and how far we can trust these findings.

1. Why do we "hear" a sound that was coughed out of a recording?

In an experiment from 1970, a recording of the English word "legislatures" was played. The first "s" sound, which lasts about 120 milliseconds (0.12 seconds), was replaced by the sound of a cough. Of the 20 people who listened, 19 did not notice that the sound was missing [1]. This means 95 percent of the participants missed the gap. This phenomenon is called "phonemic restoration."

Is this just a trick of the mind, or is the brain actually rebuilding the sound? A study in 2016 investigated this by placing electrodes on the brains of five people. When a part of a word was replaced by noise, the activity in the auditory cortex (the part of the brain that processes sound) matched the sound the person said they heard, not the noise they actually received [2]. The researchers interpreted this as the auditory cortex restoring the speech. The difference in brain activity appeared about 150 milliseconds (0.15 seconds) after the sound started [2]. A Japanese summary of this research also notes that the auditory cortex fills in sounds hidden by noise in real time [3].

However, this study only recorded data from five people. We cannot say for certain that every human brain works exactly this way. Still, it shows that "hearing" is not just a passive record of what reaches the ear. It is also a job for the brain. So, how does the brain choose which sound to focus on when many sounds overlap?

2. Can you really hear your own name in a noisy place?

In places like parties, where many people are talking, you can still hear the person speaking to you. In 1953, Colin Cherry named this challenge the "cocktail party problem" [4]. Cherry conducted an experiment where participants heard two voices at once and repeated one of them. He found that it is easier to separate voices if they differ in gender, direction, pitch, or speed of speech [4].

It is often said that "you always hear your name even in a noisy crowd." It is true that words you are interested in can be heard clearly [6]. However, an experiment by Moray in 1959 showed that many people did not notice their own name when it was played in the ear they were not paying attention to, according to textbook summaries [5]. Some people did notice it, but it was not guaranteed. So, it is not true that you will always hear your name.

In a university lecture, it is explained that the brain selects sounds using instructions from the prefrontal cortex (which directs attention) and signals rising from the ears. In the thalamus, a substance called acetylcholine is said to help align the timing of these signals [6]. This explanation is still at the research stage and is not fully settled. The brain selects sounds, fills in gaps, and extracts meaning. Can it also read things beyond the words themselves, like emotions?

3. Can you tell that someone is smiling just from their voice?

Have you ever felt that the person on the other end of the phone was smiling? In 1980, an experiment by Tartter recorded six speakers saying 29 sentences. They said them twice: once with a smile and once with a neutral face. Listeners were asked to guess which version was which. For every speaker, the listeners guessed correctly more often than chance would allow [7].

When people smile, the pitch (fundamental frequency) of their voice and the resonance frequencies that shape the voice's quality go up [7]. The idea is that smiling changes the shape of the vocal tract, which changes the voice itself. The conclusion of this study is that a smile can be heard, not just seen [7].

However, this was an experiment with only six speakers. It does not mean every smile can be detected. Still, it shows that people can sometimes extract information about "smiling" from the voice, even though the sound waves do not explicitly say "I am smiling." What about more complex emotions? Can feelings be conveyed through voices in languages you do not know?

4. Can we feel emotions in voices speaking languages we don't know?

In a 2018 study, 80 people (children aged 8 to 13 and adults who spoke English) listened to voices in English, Spanish, Chinese, and Arabic. They had to choose one emotion from five options: anger, joy, sadness, fear, or neutral [8]. With five choices, random guessing would be correct 20 percent of the time. The results showed that for every language, people were more accurate than random guessing. However, they were more accurate with their native language, English [8]. A Japanese article on this research notes that recognizing anger and sadness is more accurate, and this ability grows significantly from adolescence to adulthood [10].

Because the participants in this study spoke English, we cannot say that children in Japan would have the same results. Another study in 2018 involved 225 people aged 5 to 17 and 30 adults. They chose emotions (anger, fear, joy, sadness) from short voice clips. Their performance approached adult levels around age 14 or 15 [9]. In the tasks of these two studies, scores improved with age, though the rate of growth varied by language and task.

So, the ear and brain fill in missing sounds, pick out voices, and read emotions. How can you test this with your own voice?

5. How can you test whether a smile can be heard?

Ask a family member or friend to turn their back to you. Say the same words (like "Hello" or "Thank you") twice with a neutral face and twice while smiling. Change the order and ask them to guess which times you were smiling. If they guess correctly, ask them, "What made you think so?" It might be the pitch of the voice or its resonance. Answers may vary. If they guess wrong, remember that one try does not decide anything. Tartter’s experiment used six speakers and 29 sentences to show that results were better than chance [7].

Another fun way to explore this is to listen to voices on TV or radio without looking at the screen. Try to imagine what face the speaker is making.

The numbers in this article come from secondary sources. If you want to know more, you can look up the papers by Leonard et al. [2], Chronaki et al. [8], and Grosbras et al. [9] in the reference list to see the number of participants and conditions for yourself.

Sources

  1. Wikipedia, "Phonemic restoration effect" https://en.wikipedia.org/wiki/Phonemic_restoration_effect (Introduces the experiment where a cough replaced a sound.)
  2. Leonard, M. K. et al., Nature Communications 7:13619, 2016 https://pmc.ncbi.nlm.nih.gov/articles/PMC5187421/ (Confirms brain restoration using surface electrodes.)
  3. Nature Asia Highlight https://www.natureasia.com/ja-jp/research/highlight/11492 (Japanese summary of the brain restoration study.)
  4. Wikipedia, "Cocktail party effect" https://en.wikipedia.org/wiki/Cocktail_party_effect (Introduces Cherry's 1953 naming and experiment.)
  5. CUNY, Sensation and Perception, Selective Attention https://pressbooks.cuny.edu/sensationandperception/chapter/selective-attention/ (Summarizes Moray's 1959 name experiment.)
  6. https://telemail.jp/shingaku/academics-research/lecture/g015292 ()
  7. Tartter, V. C., Perception & Psychophysics 27:24-27, 1980 https://link.springer.com/article/10.3758/BF03199901 (Summary of the smiling voice experiment.)
  8. Chronaki, G. et al., Scientific Reports 8:8659, 2018 https://cris.maastrichtuniversity.nl/en/publications/the-development-of-cross-cultural-recognition-of-vocal-emotion-du/ (Experiment identifying emotions in four languages.)
  9. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6172235/ ()
  10. Nature Asia Highlight https://www.natureasia.com/ja-jp/research/highlight/12557 (Japanese summary of Chronaki's research.)