Publikation:

More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension

Lade...
Vorschaubild

Dateien

Zhang_2-1pekl5l7fcd8p3.pdf
Zhang_2-1pekl5l7fcd8p3.pdfGröße: 1.03 MBDownloads: 116

Datum

2021

Autor:innen

Zhang, Ye
Tuomainen, Jyrki
Skipper, Jeremy I.
Vigliocco, Gabriella

Herausgeber:innen

Kontakt

ISSN der Zeitschrift

Electronic ISSN

ISBN

Bibliografische Daten

Verlag

Schriftenreihe

Auflagebezeichnung

ArXiv-ID

Internationale Patentnummer

Angaben zur Forschungsförderung

Projekt

Open Access-Veröffentlichung
Open Access Hybrid
Core Facility der Universität Konstanz

Gesperrt bis

Titel in einer weiteren Sprache

Publikationstyp
Zeitschriftenartikel
Publikationsstatus
Published

Erschienen in

Proceedings of the Royal Society of London, Series B : Biological Sciences. Royal Society of London. 2021, 288(1955), 20210500. ISSN 0962-8452. eISSN 1471-2954. Available under: doi: 10.1098/rspb.2021.0500

Zusammenfassung

The ecology of human language is face-to-face interaction, comprising cues such as prosody, co-speech gestures and mouth movements. Yet, the multimodal context is usually stripped away in experiments as dominant paradigms focus on linguistic processing only. In two studies we presented video-clips of an actress producing naturalistic passages to participants while recording their electroencephalogram. We quantified multimodal cues (prosody, gestures, mouth movements) and measured their effect on a well-established electroencephalographic marker of processing load in comprehension (N400). We found that brain responses to words were affected by informativeness of co-occurring multimodal cues, indicating that comprehension relies on linguistic and non-linguistic cues. Moreover, they were affected by interactions between the multimodal cues, indicating that the impact of each cue dynamically changes based on the informativeness of other cues. Thus, results show that multimodal cues are integral to comprehension, hence, our theories must move beyond the limited focus on speech and linguistic processing.

Zusammenfassung in einer weiteren Sprache

Fachgebiet (DDC)
400 Sprachwissenschaft, Linguistik

Schlagwörter

Konferenz

Rezension
undefined / . - undefined, undefined

Forschungsvorhaben

Organisationseinheiten

Zeitschriftenheft

Zugehörige Datensätze in KOPS

Zitieren

ISO 690ZHANG, Ye, Diego FRASSINELLI, Jyrki TUOMAINEN, Jeremy I. SKIPPER, Gabriella VIGLIOCCO, 2021. More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension. In: Proceedings of the Royal Society of London, Series B : Biological Sciences. Royal Society of London. 2021, 288(1955), 20210500. ISSN 0962-8452. eISSN 1471-2954. Available under: doi: 10.1098/rspb.2021.0500
BibTex
@article{Zhang2021words-54568,
  year={2021},
  doi={10.1098/rspb.2021.0500},
  title={More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension},
  number={1955},
  volume={288},
  issn={0962-8452},
  journal={Proceedings of the Royal Society of London, Series B : Biological Sciences},
  author={Zhang, Ye and Frassinelli, Diego and Tuomainen, Jyrki and Skipper, Jeremy I. and Vigliocco, Gabriella},
  note={Article Number: 20210500}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/54568">
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/54568"/>
    <dc:language>eng</dc:language>
    <dc:contributor>Frassinelli, Diego</dc:contributor>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dc:creator>Tuomainen, Jyrki</dc:creator>
    <dc:rights>terms-of-use</dc:rights>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/45"/>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/54568/1/Zhang_2-1pekl5l7fcd8p3.pdf"/>
    <dc:contributor>Vigliocco, Gabriella</dc:contributor>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <dc:contributor>Skipper, Jeremy I.</dc:contributor>
    <dcterms:issued>2021</dcterms:issued>
    <dcterms:abstract xml:lang="eng">The ecology of human language is face-to-face interaction, comprising cues such as prosody, co-speech gestures and mouth movements. Yet, the multimodal context is usually stripped away in experiments as dominant paradigms focus on linguistic processing only. In two studies we presented video-clips of an actress producing naturalistic passages to participants while recording their electroencephalogram. We quantified multimodal cues (prosody, gestures, mouth movements) and measured their effect on a well-established electroencephalographic marker of processing load in comprehension (N400). We found that brain responses to words were affected by informativeness of co-occurring multimodal cues, indicating that comprehension relies on linguistic and non-linguistic cues. Moreover, they were affected by interactions between the multimodal cues, indicating that the impact of each cue dynamically changes based on the informativeness of other cues. Thus, results show that multimodal cues are integral to comprehension, hence, our theories must move beyond the limited focus on speech and linguistic processing.</dcterms:abstract>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/45"/>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/54568/1/Zhang_2-1pekl5l7fcd8p3.pdf"/>
    <dc:creator>Vigliocco, Gabriella</dc:creator>
    <dc:contributor>Tuomainen, Jyrki</dc:contributor>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-08-12T07:05:14Z</dcterms:available>
    <dc:creator>Frassinelli, Diego</dc:creator>
    <dc:creator>Skipper, Jeremy I.</dc:creator>
    <dc:contributor>Zhang, Ye</dc:contributor>
    <dcterms:title>More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension</dcterms:title>
    <dc:creator>Zhang, Ye</dc:creator>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-08-12T07:05:14Z</dc:date>
  </rdf:Description>
</rdf:RDF>

Interner Vermerk

xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter

Kontakt
URL der Originalveröffentl.

Prüfdatum der URL

Prüfungsdatum der Dissertation

Finanzierungsart

Kommentar zur Publikation

Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Ja
Diese Publikation teilen