The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports

Vorschaubild nicht verfügbar
Dateien
Zu diesem Dokument gibt es keine Dateien.
Datum
2023
Autor:innen
Leuschner, Elena
Burton, Ian
Kerl, Tilman
Herausgeber:innen
Kontakt
ISSN der Zeitschrift
eISSN
item.preview.dc.identifier.isbn
Bibliografische Daten
Verlag
Schriftenreihe
URI (zitierfähiger Link)
DOI (zitierfähiger Link)
ArXiv-ID
Internationale Patentnummer
Link zur Lizenz
EU-Projektnummer
Projekt
Open Access-Veröffentlichung
Gesperrt bis
Titel in einer weiteren Sprache
Forschungsvorhaben
Organisationseinheiten
Zeitschriftenheft
Publikationstyp
Zeitschriftenartikel
Publikationsstatus
Published
Erschienen in
The Review of International Organizations ; 2023. - Springer. - ISSN 1559-7431. - eISSN 1559-744X
Zusammenfassung
International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.
Zusammenfassung in einer weiteren Sprache
Fachgebiet (DDC)
320 Politik
Schlagwörter
Konferenz
Rezension
undefined / . - undefined, undefined. - (undefined; undefined)
Zitieren
ISO 690ECKHARD, Steffen, Vytautas JANKAUSKAS, Elena LEUSCHNER, Ian BURTON, Tilman KERL, Rita SEVASTJANOVA, 2023. The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports. In: The Review of International Organizations. Springer. ISSN 1559-7431. eISSN 1559-744X. Available under: doi: 10.1007/s11558-023-09489-1
BibTex
@article{Eckhard2023perfo-66865,
  year={2023},
  doi={10.1007/s11558-023-09489-1},
  title={The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports},
  issn={1559-7431},
  journal={The Review of International Organizations},
  author={Eckhard, Steffen and Jankauskas, Vytautas and Leuschner, Elena and Burton, Ian and Kerl, Tilman and Sevastjanova, Rita}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/66865">
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:creator>Kerl, Tilman</dc:creator>
    <dcterms:abstract>International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.</dcterms:abstract>
    <dc:contributor>Kerl, Tilman</dc:contributor>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-05-11T07:39:17Z</dc:date>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/66865"/>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dcterms:issued>2023</dcterms:issued>
    <dc:creator>Burton, Ian</dc:creator>
    <dc:contributor>Jankauskas, Vytautas</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:contributor>Eckhard, Steffen</dc:contributor>
    <dc:creator>Eckhard, Steffen</dc:creator>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-05-11T07:39:17Z</dcterms:available>
    <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by/4.0/"/>
    <dc:creator>Sevastjanova, Rita</dc:creator>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/>
    <dc:creator>Leuschner, Elena</dc:creator>
    <dc:contributor>Sevastjanova, Rita</dc:contributor>
    <dc:contributor>Burton, Ian</dc:contributor>
    <dc:language>eng</dc:language>
    <dc:rights>Attribution 4.0 International</dc:rights>
    <dcterms:title>The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports</dcterms:title>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/>
    <dc:contributor>Leuschner, Elena</dc:contributor>
    <dc:creator>Jankauskas, Vytautas</dc:creator>
  </rdf:Description>
</rdf:RDF>
Interner Vermerk
xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter
Kontakt
URL der Originalveröffentl.
Prüfdatum der URL
Prüfungsdatum der Dissertation
Finanzierungsart
Kommentar zur Publikation
Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Begutachtet
Ja
Online First: Zeitschriftenartikel, die schon vor ihrer Zuordnung zu einem bestimmten Zeitschriftenheft (= Issue) online gestellt werden. Online First-Artikel werden auf der Homepage des Journals in der Verlagsfassung veröffentlicht.