Publikation:

BERT meets LIWC : Exploring State-of-the-Art Language Models for Predicting Communication Behavior in Couples’ Conflict Interactions

Lade...
Vorschaubild

Dateien

Biggiogera_2-1cm66gqqceoqc0.pdf
Biggiogera_2-1cm66gqqceoqc0.pdfGröße: 99.11 KBDownloads: 130

Datum

2021

Autor:innen

Biggiogera, Jacopo
Boateng, George
Hilpert, Peter
Vowels, Matthew
Bodenmann, Guy
Neysari, Mona
Kowatsch, Tobias

Herausgeber:innen

Kontakt

ISSN der Zeitschrift

Electronic ISSN

ISBN

Bibliografische Daten

Verlag

Schriftenreihe

Auflagebezeichnung

ArXiv-ID

Internationale Patentnummer

Angaben zur Forschungsförderung

Projekt

Open Access-Veröffentlichung
Open Access Green
Core Facility der Universität Konstanz

Gesperrt bis

Titel in einer weiteren Sprache

Publikationstyp
Beitrag zu einem Konferenzband
Publikationsstatus
Published

Erschienen in

HAMMAL, Zakia, ed., Carlo BUSSO, ed., Catherine PELACHAUD, ed. and others. ICMI '21 Companion : Companion Publication of the 2021 International Conference on Multimodal Interaction. New York, NY: ACM, 2021, pp. 385-389. ISBN 978-1-4503-8471-1. Available under: doi: 10.1145/3461615.3485423

Zusammenfassung

Many processes in psychology are complex, such as dyadic interactions between two interacting partners (e.g., patient-therapist, intimate relationship partners). Nevertheless, many basic questions about interactions are difficult to investigate because dyadic processes can be within a person and between partners, they are based on multimodal aspects of behavior and unfold rapidly. Current analyses are mainly based on the behavioral coding method, whereby human coders annotate behavior based on a coding schema. But coding is labor-intensive, expensive, slow, focuses on few modalities, and produces sparse data which has forced the field to use average behaviors across entire interactions, thereby undermining the ability to study processes on a fine-grained scale. Current approaches in psychology use LIWC for analyzing couples’ interactions. However, advances in natural language processing such as BERT could enable the development of systems to potentially automate behavioral coding, which in turn could substantially improve psychological research. In this work, we train machine learning models to automatically predict positive and negative communication behavioral codes of 368 German-speaking Swiss couples during an 8-minute conflict interaction on a fine-grained scale (10-seconds sequences) using linguistic features and paralinguistic features derived with openSMILE. Our results show that both simpler TF-IDF features as well as more complex BERT features performed better than LIWC, and that adding paralinguistic features did not improve the performance. These results suggest it might be time to consider modern alternatives to LIWC, the de facto linguistic features in psychology, for prediction tasks in couples research. This work is a further step towards the automated coding of couples’ behavior which could enhance couple research and therapy, and be utilized for other dyadic interactions as well.

Zusammenfassung in einer weiteren Sprache

Fachgebiet (DDC)
004 Informatik

Schlagwörter

Couples, predicting observer ratings, multimodal fusion, behavioral signal processing, BERT, LIWC, SVM

Konferenz

ICMI '21 : 2021 International Conference on Multimodal Interaction, 18. Okt. 2021 - 22. Okt. 2021, Montreal, QC, Canada
Rezension
undefined / . - undefined, undefined

Forschungsvorhaben

Organisationseinheiten

Zeitschriftenheft

Zugehörige Datensätze in KOPS

Zitieren

ISO 690BIGGIOGERA, Jacopo, George BOATENG, Peter HILPERT, Matthew VOWELS, Guy BODENMANN, Mona NEYSARI, Fridtjof W. NUSSBECK, Tobias KOWATSCH, 2021. BERT meets LIWC : Exploring State-of-the-Art Language Models for Predicting Communication Behavior in Couples’ Conflict Interactions. ICMI '21 : 2021 International Conference on Multimodal Interaction. Montreal, QC, Canada, 18. Okt. 2021 - 22. Okt. 2021. In: HAMMAL, Zakia, ed., Carlo BUSSO, ed., Catherine PELACHAUD, ed. and others. ICMI '21 Companion : Companion Publication of the 2021 International Conference on Multimodal Interaction. New York, NY: ACM, 2021, pp. 385-389. ISBN 978-1-4503-8471-1. Available under: doi: 10.1145/3461615.3485423
BibTex
@inproceedings{Biggiogera2021meets-57626,
  year={2021},
  doi={10.1145/3461615.3485423},
  title={BERT meets LIWC : Exploring State-of-the-Art Language Models for Predicting Communication Behavior in Couples’ Conflict Interactions},
  isbn={978-1-4503-8471-1},
  publisher={ACM},
  address={New York, NY},
  booktitle={ICMI '21 Companion : Companion Publication of the 2021 International Conference on Multimodal Interaction},
  pages={385--389},
  editor={Hammal, Zakia and Busso, Carlo and Pelachaud, Catherine},
  author={Biggiogera, Jacopo and Boateng, George and Hilpert, Peter and Vowels, Matthew and Bodenmann, Guy and Neysari, Mona and Nussbeck, Fridtjof W. and Kowatsch, Tobias}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/57626">
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/57626"/>
    <dc:creator>Hilpert, Peter</dc:creator>
    <dc:contributor>Vowels, Matthew</dc:contributor>
    <dc:contributor>Nussbeck, Fridtjof W.</dc:contributor>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/43"/>
    <dc:language>eng</dc:language>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dc:contributor>Kowatsch, Tobias</dc:contributor>
    <dcterms:abstract xml:lang="eng">Many processes in psychology are complex, such as dyadic interactions between two interacting partners (e.g., patient-therapist, intimate relationship partners). Nevertheless, many basic questions about interactions are difficult to investigate because dyadic processes can be within a person and between partners, they are based on multimodal aspects of behavior and unfold rapidly. Current analyses are mainly based on the behavioral coding method, whereby human coders annotate behavior based on a coding schema. But coding is labor-intensive, expensive, slow, focuses on few modalities, and produces sparse data which has forced the field to use average behaviors across entire interactions, thereby undermining the ability to study processes on a fine-grained scale. Current approaches in psychology use LIWC for analyzing couples’ interactions. However, advances in natural language processing such as BERT could enable the development of systems to potentially automate behavioral coding, which in turn could substantially improve psychological research. In this work, we train machine learning models to automatically predict positive and negative communication behavioral codes of 368 German-speaking Swiss couples during an 8-minute conflict interaction on a fine-grained scale (10-seconds sequences) using linguistic features and paralinguistic features derived with openSMILE. Our results show that both simpler TF-IDF features as well as more complex BERT features performed better than LIWC, and that adding paralinguistic features did not improve the performance. These results suggest it might be time to consider modern alternatives to LIWC, the de facto linguistic features in psychology, for prediction tasks in couples research. This work is a further step towards the automated coding of couples’ behavior which could enhance couple research and therapy, and be utilized for other dyadic interactions as well.</dcterms:abstract>
    <dc:creator>Vowels, Matthew</dc:creator>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/57626/1/Biggiogera_2-1cm66gqqceoqc0.pdf"/>
    <dc:creator>Biggiogera, Jacopo</dc:creator>
    <dc:creator>Kowatsch, Tobias</dc:creator>
    <dc:contributor>Hilpert, Peter</dc:contributor>
    <dc:creator>Bodenmann, Guy</dc:creator>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/57626/1/Biggiogera_2-1cm66gqqceoqc0.pdf"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/43"/>
    <dc:contributor>Biggiogera, Jacopo</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:creator>Boateng, George</dc:creator>
    <dc:contributor>Bodenmann, Guy</dc:contributor>
    <dc:creator>Neysari, Mona</dc:creator>
    <dcterms:issued>2021</dcterms:issued>
    <dcterms:title>BERT meets LIWC : Exploring State-of-the-Art Language Models for Predicting Communication Behavior in Couples’ Conflict Interactions</dcterms:title>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2022-05-23T12:21:35Z</dcterms:available>
    <dc:rights>terms-of-use</dc:rights>
    <dc:contributor>Neysari, Mona</dc:contributor>
    <dc:creator>Nussbeck, Fridtjof W.</dc:creator>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2022-05-23T12:21:35Z</dc:date>
    <dc:contributor>Boateng, George</dc:contributor>
  </rdf:Description>
</rdf:RDF>

Interner Vermerk

xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter

Kontakt
URL der Originalveröffentl.

Prüfdatum der URL

Prüfungsdatum der Dissertation

Finanzierungsart

Kommentar zur Publikation

Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Diese Publikation teilen