KOPS - The Institutional Repository of the University of Konstanz

More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension

More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension

Cite This

Files in this item

Checksum: MD5:f428bb37b09a125b366f1bf34d7272c0

ZHANG, Ye, Diego FRASSINELLI, Jyrki TUOMAINEN, Jeremy I. SKIPPER, Gabriella VIGLIOCCO, 2021. More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension. In: Proceedings of the Royal Society of London, Series B : Biological Sciences. Royal Society of London. 288(1955), 20210500. ISSN 0962-8452. eISSN 1471-2954. Available under: doi: 10.1098/rspb.2021.0500

@article{Zhang2021words-54568, title={More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension}, year={2021}, doi={10.1098/rspb.2021.0500}, number={1955}, volume={288}, issn={0962-8452}, journal={Proceedings of the Royal Society of London, Series B : Biological Sciences}, author={Zhang, Ye and Frassinelli, Diego and Tuomainen, Jyrki and Skipper, Jeremy I. and Vigliocco, Gabriella}, note={Article Number: 20210500} }

<rdf:RDF xmlns:dcterms="http://purl.org/dc/terms/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:bibo="http://purl.org/ontology/bibo/" xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:void="http://rdfs.org/ns/void#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > <rdf:Description rdf:about="https://kops.uni-konstanz.de/rdf/resource/123456789/54568"> <dc:creator>Skipper, Jeremy I.</dc:creator> <dcterms:title>More than words : word predictability, prosody, gesture and mouth movements in natural language comprehension</dcterms:title> <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/54568/1/Zhang_2-1pekl5l7fcd8p3.pdf"/> <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/> <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/54568"/> <dc:creator>Vigliocco, Gabriella</dc:creator> <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/54568/1/Zhang_2-1pekl5l7fcd8p3.pdf"/> <dc:creator>Frassinelli, Diego</dc:creator> <dc:creator>Tuomainen, Jyrki</dc:creator> <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/> <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-08-12T07:05:14Z</dcterms:available> <dcterms:abstract xml:lang="eng">The ecology of human language is face-to-face interaction, comprising cues such as prosody, co-speech gestures and mouth movements. Yet, the multimodal context is usually stripped away in experiments as dominant paradigms focus on linguistic processing only. In two studies we presented video-clips of an actress producing naturalistic passages to participants while recording their electroencephalogram. We quantified multimodal cues (prosody, gestures, mouth movements) and measured their effect on a well-established electroencephalographic marker of processing load in comprehension (N400). We found that brain responses to words were affected by informativeness of co-occurring multimodal cues, indicating that comprehension relies on linguistic and non-linguistic cues. Moreover, they were affected by interactions between the multimodal cues, indicating that the impact of each cue dynamically changes based on the informativeness of other cues. Thus, results show that multimodal cues are integral to comprehension, hence, our theories must move beyond the limited focus on speech and linguistic processing.</dcterms:abstract> <dc:contributor>Frassinelli, Diego</dc:contributor> <dcterms:issued>2021</dcterms:issued> <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/rdf/resource/123456789/45"/> <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-08-12T07:05:14Z</dc:date> <dc:contributor>Vigliocco, Gabriella</dc:contributor> <foaf:homepage rdf:resource="http://localhost:8080/jspui"/> <dc:language>eng</dc:language> <dc:rights>terms-of-use</dc:rights> <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/rdf/resource/123456789/45"/> <dc:creator>Zhang, Ye</dc:creator> <dc:contributor>Skipper, Jeremy I.</dc:contributor> <dc:contributor>Tuomainen, Jyrki</dc:contributor> <dc:contributor>Zhang, Ye</dc:contributor> </rdf:Description> </rdf:RDF>

Downloads since Aug 12, 2021 (Information about access statistics)

Zhang_2-1pekl5l7fcd8p3.pdf 13

This item appears in the following Collection(s)

Search KOPS


Browse

My Account