Language

ž  Language

ž  Speech Recognition and Processing

ž  The BonOBOS – Video Clip

ž  The Bonobo Brain

ž  From ape to human. Magnetic resonance images of a bonobo brain are warped onto the shape of a human cortex, viewed from (left to right) the side, top, and front. Red and yellow areas in the temporal region (linked to language) and in the prefrontal and occipital regions had to be stretched the most to reach the human configuration, whereas blue areas are similar in apes and humans

ž  Note that Brodmann’s 44 is enlarged in apes & bonobos.

ž  A Model of Speech Comprehension and Production

ž  This lecture concerns processes on the right of this diagram

ž  How do we process sentences?

Sometimes the context helps:

Please pass me the book on the table.

Often the context does not help:

Giant lizard-like creatures are descending

from spaceships and attacking Tableview.

A body-building Austrian nicknamed “The

Terminator” will be elected to a major political office.

ž  The Nature of the Acoustic Signal

·         Spectogram shows frequency against time, intensity shown by darkness

ž  The Nature of the Acoustic Signal

ž  Gaps do not occur between words but occur with certain consonants (that restrict flow of air)

ž  Problem of segmenting continuous input (ice cream v. I scream)

ž  Problem of inter-speaker differences; pitch affected by age and sex; different dialects, talking speeds etc.

ž  Co-articulation = consecutive speech sounds blend into each other due to mechanical constraints on articulators

¡  E.g. “add” and “adder” (d is softer when followed by vowel)

ž  How Do Listeners Deal with Variability in Acoustic Input?

ž  Categorical perception: continuous changes in input are mapped on to discrete percepts

¡  E.g.Voicing “Da” and “Ta” do not sound the same

¡  E.g. Voicing “Add” and “Adder” sound the same

ž  These may be mapped on to abstract representations that specify nature of acoustic signal (e.g. voicing, timing), phonemes, syllables

ž  How Do Listeners Deal with Variability in Acoustic Input?

ž  Could also be mapped on to units of articulation (i.e. understanding what other people are saying by figuring out how I could say it) – motor theory of speech perception

ž  There is a link between the speech production and speech processing areas in the brain

ž  Evidence - Watkins and Paus (2004) increased motor excitability during speech perception (Transcranial Magnetic Stimulation (TMS), PET)

ž  Speech Perception in the Brain

ž  Auditory nerve passes through medial geniculate nucleus on way to primary auditory cortex (A1) in temporal lobes, Heschl’s  gyrus (gyrus = ridge, sulci = fold)

ž  Thalamus involved in processing sensory information (lesions can lead to thalamic aphasia)

ž  Tendency for different ears to project to opposite cortex

          (also some subcortical routes in hearing as in vision)

ž  Primary auditory cortex is tonotopically organised (i.e. organised according to the frequency of sound to which they respond best)

ž  Neuroanatomy

ž  Hierarchical brain systems for
Word recognition:

ž  First, the stream of auditory information proceeds from auditory cortex in Heschl’s gyri to the superior temporal gyrus (STG). Here, no distinction is made between speech and non-speech sounds.

ž  Distinction is made between speech and non-speech sounds in the adjacent superior temporal sulcus (STS), but no lexical-semantic information is processed in this area.

ž  Hierarchical brain systems for
Word recognition:

ž  From the STS, the information proceeds to the middle and inferior temporal gyri, where phonological and lexical-semantic aspect of word is processed.

ž  The next stage involves analysis in the angular gyrus (involved in naming).

ž  Broca area may be important for processing syntactic information.

ž  Another area for syntactic processing is area 22 in STG.

ž                                  Temporal LOBE

ž  Superior Temporal Sulcus (orange)

¡  divides the superior temporal gyrus (peach) from middle temporal gyrus (lime)

ž  Inferior Temporal Sulcus (blue)

¡  not usually very continuous

¡  divides middle temporal gyrus from inferior temporal gyrus (lavender)

ž  Naming of objects not arbitrary

ž  A remote tribe calls one of these shapes Booba

          and the other Kiki. Decide which is which.

ž  Connecting visual stimuli to language

ž  Abnormalities in FG involved with Synesthesia

          (e.g. seeing music), also out-of-body experiences

ž  Speech Perception in the Brain

ž  Primary auditory cortex responds equally to speech and other sounds in both left and right hemispheres – lesions result in a loss of awareness of sound, but patients can still react reflexively to sound

ž  Areas more anterior to this in left hemisphere respond more to intelligible speech relative to unintelligible speech of similar acoustic complexity (Scott et al., 2000)

ž  Left hemisphere damage can result in a type of auditory agnosia (pure word deafness) in which environmental sounds and music are identified, but not speech – speech appears "too fast" or is “distorted”

ž  Disconnection theory proposes that inputs from both Heschl gyri are cut off from input into the left hemisphere Wernicke’s area where sounds are decoded into language.

ž  Dual Routes for Speech Perception: "What" and "How"

·         As with vision, it has been suggested that speech perception has 2 functionally distinct pathways

·         Phonological buffer refers to holding phonologically coded information but capable of maintaining the information only for brief periods (think of STM)

ž  Dual Routes for Speech Perception: "What" and "How"

ž  "What" route…

¡  Ventral route along temporal lobe

¡  Recognizes speech acoustically

¡  Important for speech comprehension (i.e.

        makes contact with semantic knowledge)

        green = dorsal (back)

        purple = ventral (bottom)

ž  Dual Routes for Speech Perception: "What" and "How"

ž  "How" route…

¡  Dorsal route involving parieto-frontal circuit

¡  Recognizes speech motorically (i.e. motor theory of speech perception)

¡  Used to say and learn unfamiliar words

¡  Part of Wernicke’s area responds to silent articulation (by speaker) and also viewing lip movements in others (Wise et al.)

¡  Evidence for phonological STM in angular gyrus (Paulesu et al., 1993), which may be refreshed by frontal rehearsal mechanisms (e.g. phonological loop component of working memory)

ž  Dual Routes for Speech Perception: "What“ (ventral) and "How“ (Dorsal)

ž  Deficits in repeating and learning new phonology linked to phonological STM impairments

¡  = deficit in "how" route, intact "what"?

ž  Patients with deep dysphasia cannot repeat non-words and make semantic errors in repetition (e.g. hear "cat", say "dog")

¡  = deficit in "how" route, rely on impoverished "what"?

ž  Language

ž  Speech Processing

ž  Recognizing Spoken Words: The Cohort Model

ž  Recognizing Spoken Words: The Cohort Model

ž  All candidates considered in parallel

ž  Candidates eliminated as more evidence becomes available in the speech input

ž  Uniqueness point occurs when only one candidate remains

ž  Reaction time priming shows that uncommon words activated less (speed > species)

ž  However, semantic context does not alter the pattern

ž  Suggests semantics occurs late (i.e. after spoken word recognition)

ž  Recognizing Spoken Words: The Cohort Model

·         Evidence for a late influence of semantics comes from N400 in ERPs (Event Related Potentials in Electrophysiological studies)

ž  Putting Words into Sentences: Role of Syntax and Semantics

ž  Parsing = putting words into sentences

ž  A & B have different meaning but same syntax

ž  A & C have same meaning but different syntax

A = The boy hit the girl

B = The girl hit the boy

C = The girl was hit by the boy

ž  Putting Words into Sentences: Role of Syntax and Semantics

ž  When parsing a sentence are all possible sentence constructions considered in parallel or is just one syntactic structure considered?  Read this…

¡  "The fireman told the man that he had risked his life for to install a smoke detector"

ž  Did it make sense? The fact that it probably didn’t implies that not all syntactic frames were considered (garden-path sentences) The early part of the sentence biases a certain syntactic interpretation that turns out to be incorrect.

ž  However, semantics can bias syntax. E.g. if preceded by “The fireman braved a dangerous fire in a hotel. He rescued one of the men at great danger to himself”

ž  Category Specificity in other domains of knowledge

ž   A Historical Preamble

·         Previously believed that Broca’s aphasia (and Broca’s area) was related to speech production and Wernicke’s area (and Wernicke’s aphasia) related to speech comprehension

ž  Broca’s and Wernicke’s area

ž  An Example of Broca’s Aphasia

·         “cookie jar… fall over… chair… water… empty…”

ž  Against the 19th-Century Model

ž  Broca’s aphasic patients also have some problems in comprehension

ž  A deficit in "motor images" doesn’t explain the main symptom (agrammatism)

ž  Many patients who meet criteria for Broca’s aphasia have damage in temporal lobes not Broca’s area (Dronkers)

ž  Wernicke’s aphasic patients also have problems in speech production (e.g. neologisms)

ž  Wernicke’s area involved in a variety of functions including linking acoustic information with visual and motoric information

ž  So What does Broca’s Area do? Compute Syntax?

·         In 1970s the view shifted from a deficit in speech production to a deficit in syntax (in both

          comprehension and production)

ž  So What does Broca’s Area do? Challenges to the Syntactic Theory

ž  In 1970s the view shifted from a deficit in speech production to a deficit in syntax (in

          both comprehension and production)

ž  This has also fallen out of favour because…

¡  Broca’s aphasia not necessarily a reliable or unitary disorder – can be agrammatic in some respects but

        not other

¡  Some Broca’s aphasics are agrammatic in production but not comprehension

¡  Sentence-processing deficits could be explained by use of Broca’s area in working memory rather than syntax

ž  So What does Broca’s Area do? The Current Consensus

ž  UNLIKELY to be a central syntactic device (although some people still believe)

ž  UNLIKELY to store actual motor programs for speech (Broca’s original idea) but may be involved in higher-level planning of speech

ž  LIKELY to be involved in linking action perception to action production (region 44), this includes speech as well as other actions (mirror neurons)

ž  LIKELY to be involved in verbal working memory

          (region 45) including establishing semantic and

          thematic coherence

ž  Language

ž  Speech Production and Retrieval

ž  Retrieving Spoken Words

ž  After syntactic and semantic aspects of utterance are put in place, speaker must select the individual words comprising the utterance

ž  Lexicalization = selecting a single word based on the meaning one wishes to convey (constrained by pragmatics)

ž  Grammatical properties of words must be specified (noun, verb, etc.)

ž  Word form (phonemes, syllables, etc.) must be retrieved

ž  Evidence that these occur in different stages

ž  Retrieving Spoken Words: What Matters?

ž  Studies of Speech Errors

ž  Provide evidence of separate linguistic units (words, morphemes, phonemes) because like tends to substitute for like

ž  Semantic errors: say "dog", intend "cat" (evidence of competition at semantic level)

ž  Freudian slip: say "weapons of mass distraction", intend "destruction"? (Freud believed these errors revealed hidden intentions of the speaker)

ž  More Studies of Speech Errors

ž  Lexical transpositions: "guess whose mind came to name" (noun for noun) or "I randomed some samply" (stranded morphemes)

ž  Malapropisms: say "hysterical" instead of "historical" (evidence of competition at word-form level)

ž  Spoonerisms: say "hissed mystery lectures", intend "missed history lectures" (phonemes in same position swapped)

ž  Tip-of-tongue Phenomenon

ž  Speaker knows, conceptually, intended word but is unable to produce spoken output

ž  Accompanied by "feeling of knowing" and very frustrating

ž  More common in low frequency words (and in older people)

ž  Speakers often know first letter, length of word, etc.

ž  Italian speakers often know word gender

ž  Suggests that spoken word retrieval occurs in chunks rather than "all or nothing"

ž  Levelt’s Discrete Stages Model

ž  Levelt’s Discrete Stages Model: Evidence in Favour

ž  Explains TOT as retrieval of lemma with partial or no retrieval of lexeme

ž  Explains patients with anomia (pathological word finding problems), which can arise at two stages – choosing a concept (accompanied by deficits in semantic memory) versus choosing a word (intact semantic memory)

ž  Discrete stages because…

¡  "sheep" primes "goat" (competition at semantic level)

¡  "goat" primes "goal" (competition at phonological level)

¡   "sheep" does NOT prime "goal" (suggests 2 stages of

        competition do not interact)

ž  Levelt’s Discrete Stages Model: Evidence Against

ž  High proportion of mixed errors (with both semantic

          and phonological characteristics) suggests interaction between levels? E.g.

           "rat" for cat, "oyster" for lobster

ž  Caramazza and Miozzo: in TOT possible to retrieve gender without first letter (lemma access without lexeme access) but also

          possible to retrieve first letter without

          gender (lexeme access without

          lemma access). The latter is not allowed in

          Levelt’s model

ž  Dell’s Interactive Stages Model

·         Similar stages to Levelt's but interactivity explains

          high proportion of mixed errors

ž  Articulating an Utterance

ž  Patients with articulation problems have lesions in insula and basal ganglia but not Broca’s area (Dronkers, 1996)

ž  This is called apraxia for speech (can sound like a foreign accent sometimes)

ž  fMRI of articulation relative to speech perception also activates insula and frontal-motor regions but not Broca’s area (Wise et al., 1999)

ž  Others (e.g. Indefrey & Levelt) suggest that Broca’s area is, however, involved in overt and covert planning of speech production even if motor commands do not reside there – evidence for mirror neurons here could be consistent with this