Publikation: The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports
Dateien
Datum
Autor:innen
Herausgeber:innen
ISSN der Zeitschrift
Electronic ISSN
ISBN
Bibliografische Daten
Verlag
Schriftenreihe
Auflagebezeichnung
URI (zitierfähiger Link)
DOI (zitierfähiger Link)
Internationale Patentnummer
Link zur Lizenz
Angaben zur Forschungsförderung
Projekt
Open Access-Veröffentlichung
Core Facility der Universität Konstanz
Titel in einer weiteren Sprache
Publikationstyp
Publikationsstatus
Erschienen in
Zusammenfassung
International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.
Zusammenfassung in einer weiteren Sprache
Fachgebiet (DDC)
Schlagwörter
Konferenz
Rezension
Zitieren
ISO 690
ECKHARD, Steffen, Vytautas JANKAUSKAS, Elena LEUSCHNER, Ian BURTON, Tilman KERL, Rita SEVASTJANOVA, 2023. The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports. In: The Review of International Organizations. Springer. 2023, 18(4), S. 753-776. ISSN 1559-7431. eISSN 1559-744X. Verfügbar unter: doi: 10.1007/s11558-023-09489-1BibTex
@article{Eckhard2023-05-06perfo-66865, title={The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports}, year={2023}, doi={10.1007/s11558-023-09489-1}, number={4}, volume={18}, issn={1559-7431}, journal={The Review of International Organizations}, pages={753--776}, author={Eckhard, Steffen and Jankauskas, Vytautas and Leuschner, Elena and Burton, Ian and Kerl, Tilman and Sevastjanova, Rita}, note={DFG under Grant Agreement No EC 506/2–1} }
RDF
<rdf:RDF xmlns:dcterms="http://purl.org/dc/terms/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:bibo="http://purl.org/ontology/bibo/" xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:void="http://rdfs.org/ns/void#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/66865"> <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/66865/1/Eckhard_2-mz3jdaqm1vf66.pdf"/> <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/> <dc:creator>Kerl, Tilman</dc:creator> <dcterms:abstract>International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.</dcterms:abstract> <dc:contributor>Kerl, Tilman</dc:contributor> <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-05-11T07:39:17Z</dc:date> <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/66865"/> <foaf:homepage rdf:resource="http://localhost:8080/"/> <dc:creator>Burton, Ian</dc:creator> <dc:contributor>Jankauskas, Vytautas</dc:contributor> <dcterms:issued>2023-05-06</dcterms:issued> <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/> <dc:contributor>Eckhard, Steffen</dc:contributor> <dc:creator>Eckhard, Steffen</dc:creator> <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-05-11T07:39:17Z</dcterms:available> <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by/4.0/"/> <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/> <dc:creator>Sevastjanova, Rita</dc:creator> <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/> <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/66865/1/Eckhard_2-mz3jdaqm1vf66.pdf"/> <dc:creator>Leuschner, Elena</dc:creator> <dc:contributor>Sevastjanova, Rita</dc:contributor> <dc:contributor>Burton, Ian</dc:contributor> <dc:language>eng</dc:language> <dc:rights>Attribution 4.0 International</dc:rights> <dcterms:title>The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports</dcterms:title> <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/> <dc:contributor>Leuschner, Elena</dc:contributor> <dc:creator>Jankauskas, Vytautas</dc:creator> </rdf:Description> </rdf:RDF>