Publikation:

Electrophysiological signatures of second language multimodal comprehension

Lade...
Vorschaubild

Dateien

Zu diesem Dokument gibt es keine Dateien.

Datum

2021

Autor:innen

Zhang, Ye
Ding, Rong
Tuomainen, Jyrki
Klavinskis-Whiting, Sebastian
Vigliocco, Gabriella

Herausgeber:innen

Kontakt

ISSN der Zeitschrift

Electronic ISSN

ISBN

Bibliografische Daten

Verlag

Schriftenreihe

Auflagebezeichnung

URI (zitierfähiger Link)
DOI (zitierfähiger Link)
ArXiv-ID

Internationale Patentnummer

Angaben zur Forschungsförderung

Projekt

Open Access-Veröffentlichung
Core Facility der Universität Konstanz

Gesperrt bis

Titel in einer weiteren Sprache

Publikationstyp
Beitrag zu einem Konferenzband
Publikationsstatus
Published

Erschienen in

Comparative Cognition : Animal Minds : Proceedings of the 43rd Annual Conference of the Cognitive Science Society (CogSci 2021). Vienna: Cognitive Science Society, 2021, pp. 2971-2977

Zusammenfassung

Language is multimodal: non-linguistic cues, such as prosody, gestures and mouth movements, are always present in face-toface communication and interact to support processing. In this paper, we ask whether and how multimodal cues affect L2 processing by recording EEG for highly proficient bilinguals when watching naturalistic materials. For each word, we quantified surprisal and the informativeness of prosody, gestures, and mouth movements. We found that each cue modulates the N400: prosodic accentuation, meaningful gestures, and informative mouth movements all reduce N400. Further, effects of meaningful gestures but not mouth informativeness are enhanced by prosodic accentuation, whereas effects of mouth are enhanced by meaningful gestures but reduced by beat gestures. Compared with L1, L2 participants benefit less from cues and their interactions, except for meaningful gestures and mouth movements. Thus, in realworld language comprehension, L2 comprehenders use multimodal cues just as L1 speakers albeit to a lesser extent.

Zusammenfassung in einer weiteren Sprache

Fachgebiet (DDC)
400 Sprachwissenschaft, Linguistik

Schlagwörter

multimodal communication; language; N400; gesture; mouth; prosodic accentuation, bilingualism, L2

Konferenz

43rd Annual Conference of the Cognitive Science Society (CogSci 2021), 26. Juli 2021 - 29. Juli 2021, Vienna, Austria
Rezension
undefined / . - undefined, undefined

Forschungsvorhaben

Organisationseinheiten

Zeitschriftenheft

Zugehörige Datensätze in KOPS

Zitieren

ISO 690ZHANG, Ye, Rong DING, Diego FRASSINELLI, Jyrki TUOMAINEN, Sebastian KLAVINSKIS-WHITING, Gabriella VIGLIOCCO, 2021. Electrophysiological signatures of second language multimodal comprehension. 43rd Annual Conference of the Cognitive Science Society (CogSci 2021). Vienna, Austria, 26. Juli 2021 - 29. Juli 2021. In: Comparative Cognition : Animal Minds : Proceedings of the 43rd Annual Conference of the Cognitive Science Society (CogSci 2021). Vienna: Cognitive Science Society, 2021, pp. 2971-2977
BibTex
@inproceedings{Zhang2021Elect-54388,
  year={2021},
  title={Electrophysiological signatures of second language multimodal comprehension},
  url={https://cognitivesciencesociety.org/wp-content/uploads/2021/07/cogsci21_proceedings_v4.pdf},
  publisher={Cognitive Science Society},
  address={Vienna},
  booktitle={Comparative Cognition : Animal Minds : Proceedings of the 43rd Annual Conference of the Cognitive Science Society (CogSci 2021)},
  pages={2971--2977},
  author={Zhang, Ye and Ding, Rong and Frassinelli, Diego and Tuomainen, Jyrki and Klavinskis-Whiting, Sebastian and Vigliocco, Gabriella}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/54388">
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/54388"/>
    <dc:contributor>Tuomainen, Jyrki</dc:contributor>
    <dc:creator>Frassinelli, Diego</dc:creator>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/45"/>
    <dc:contributor>Frassinelli, Diego</dc:contributor>
    <dc:creator>Klavinskis-Whiting, Sebastian</dc:creator>
    <dc:creator>Tuomainen, Jyrki</dc:creator>
    <dc:creator>Ding, Rong</dc:creator>
    <dcterms:abstract xml:lang="eng">Language is multimodal: non-linguistic cues, such as prosody, gestures and mouth movements, are always present in face-toface communication and interact to support processing. In this paper, we ask whether and how multimodal cues affect L2 processing by recording EEG for highly proficient bilinguals when watching naturalistic materials. For each word, we quantified surprisal and the informativeness of prosody, gestures, and mouth movements. We found that each cue modulates the N400: prosodic accentuation, meaningful gestures, and informative mouth movements all reduce N400. Further, effects of meaningful gestures but not mouth informativeness are enhanced by prosodic accentuation, whereas effects of mouth are enhanced by meaningful gestures but reduced by beat gestures. Compared with L1, L2 participants benefit less from cues and their interactions, except for meaningful gestures and mouth movements. Thus, in realworld language comprehension, L2 comprehenders use multimodal cues just as L1 speakers albeit to a lesser extent.</dcterms:abstract>
    <dc:rights>terms-of-use</dc:rights>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dc:language>eng</dc:language>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-07-22T12:39:15Z</dcterms:available>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <dc:contributor>Zhang, Ye</dc:contributor>
    <dc:contributor>Klavinskis-Whiting, Sebastian</dc:contributor>
    <dc:contributor>Vigliocco, Gabriella</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-07-22T12:39:15Z</dc:date>
    <dc:creator>Zhang, Ye</dc:creator>
    <dc:contributor>Ding, Rong</dc:contributor>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/45"/>
    <dc:creator>Vigliocco, Gabriella</dc:creator>
    <dcterms:title>Electrophysiological signatures of second language multimodal comprehension</dcterms:title>
    <dcterms:issued>2021</dcterms:issued>
  </rdf:Description>
</rdf:RDF>

Interner Vermerk

xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter

Kontakt

Prüfungsdatum der Dissertation

Finanzierungsart

Kommentar zur Publikation

Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Diese Publikation teilen