Publikation:

The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports

Lade...
Vorschaubild

Dateien

Eckhard_2-mz3jdaqm1vf66.pdf
Eckhard_2-mz3jdaqm1vf66.pdfGröße: 1.28 MBDownloads: 58

Datum

2023

Herausgeber:innen

Kontakt

ISSN der Zeitschrift

Electronic ISSN

ISBN

Bibliografische Daten

Verlag

Schriftenreihe

Auflagebezeichnung

ArXiv-ID

Internationale Patentnummer

Link zur Lizenz

Angaben zur Forschungsförderung

Projekt

Open Access-Veröffentlichung
Open Access Hybrid
Core Facility der Universität Konstanz

Gesperrt bis

Titel in einer weiteren Sprache

Publikationstyp
Zeitschriftenartikel
Publikationsstatus
Published

Erschienen in

The Review of International Organizations. Springer. 2023, 18(4), S. 753-776. ISSN 1559-7431. eISSN 1559-744X. Verfügbar unter: doi: 10.1007/s11558-023-09489-1

Zusammenfassung

International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.

Zusammenfassung in einer weiteren Sprache

Fachgebiet (DDC)
320 Politik

Schlagwörter

Economics and Econometrics

Konferenz

Rezension
undefined / . - undefined, undefined

Forschungsvorhaben

Organisationseinheiten

Zeitschriftenheft

Zugehörige Datensätze in KOPS

Zitieren

ISO 690ECKHARD, Steffen, Vytautas JANKAUSKAS, Elena LEUSCHNER, Ian BURTON, Tilman KERL, Rita SEVASTJANOVA, 2023. The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports. In: The Review of International Organizations. Springer. 2023, 18(4), S. 753-776. ISSN 1559-7431. eISSN 1559-744X. Verfügbar unter: doi: 10.1007/s11558-023-09489-1
BibTex
@article{Eckhard2023-05-06perfo-66865,
  title={The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports},
  year={2023},
  doi={10.1007/s11558-023-09489-1},
  number={4},
  volume={18},
  issn={1559-7431},
  journal={The Review of International Organizations},
  pages={753--776},
  author={Eckhard, Steffen and Jankauskas, Vytautas and Leuschner, Elena and Burton, Ian and Kerl, Tilman and Sevastjanova, Rita},
  note={DFG under Grant Agreement No EC 506/2–1}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/66865">
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/66865/1/Eckhard_2-mz3jdaqm1vf66.pdf"/>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:creator>Kerl, Tilman</dc:creator>
    <dcterms:abstract>International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.</dcterms:abstract>
    <dc:contributor>Kerl, Tilman</dc:contributor>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-05-11T07:39:17Z</dc:date>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/66865"/>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dc:creator>Burton, Ian</dc:creator>
    <dc:contributor>Jankauskas, Vytautas</dc:contributor>
    <dcterms:issued>2023-05-06</dcterms:issued>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:contributor>Eckhard, Steffen</dc:contributor>
    <dc:creator>Eckhard, Steffen</dc:creator>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-05-11T07:39:17Z</dcterms:available>
    <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by/4.0/"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:creator>Sevastjanova, Rita</dc:creator>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/66865/1/Eckhard_2-mz3jdaqm1vf66.pdf"/>
    <dc:creator>Leuschner, Elena</dc:creator>
    <dc:contributor>Sevastjanova, Rita</dc:contributor>
    <dc:contributor>Burton, Ian</dc:contributor>
    <dc:language>eng</dc:language>
    <dc:rights>Attribution 4.0 International</dc:rights>
    <dcterms:title>The performance of international organizations : a new measure and dataset based on computational text analysis of evaluation reports</dcterms:title>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/>
    <dc:contributor>Leuschner, Elena</dc:contributor>
    <dc:creator>Jankauskas, Vytautas</dc:creator>
  </rdf:Description>
</rdf:RDF>

Interner Vermerk

xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter

Kontakt
URL der Originalveröffentl.

Prüfdatum der URL

Prüfungsdatum der Dissertation

Finanzierungsart

Kommentar zur Publikation

DFG under Grant Agreement No EC 506/2–1
Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Ja
Link zu Forschungsdaten
Beschreibung der Forschungsdaten
All data generated and analyzed during the current study, including the language model developed as part of this study
Diese Publikation teilen