Scaling up search engine audits : Practical insights for algorithm auditing

Lade...
Vorschaubild
Dateien
Zu diesem Dokument gibt es keine Dateien.
Datum
2024
Autor:innen
Makhortykh, Mykola
Urman, Aleksandra
Herausgeber:innen
Kontakt
ISSN der Zeitschrift
Electronic ISSN
ISBN
Bibliografische Daten
Verlag
Schriftenreihe
Auflagebezeichnung
URI (zitierfähiger Link)
ArXiv-ID
Internationale Patentnummer
Angaben zur Forschungsförderung
Swiss National Science Foundation: 100001CL_182630/1
Projekt
Open Access-Veröffentlichung
Open Access Hybrid
Core Facility der Universität Konstanz
Gesperrt bis
Titel in einer weiteren Sprache
Forschungsvorhaben
Organisationseinheiten
Zeitschriftenheft
Publikationstyp
Zeitschriftenartikel
Publikationsstatus
Published
Erschienen in
Journal of Information Science. Sage. 2024, 50(2), pp. 404-419. ISSN 0165-5515. eISSN 1741-6485. Available under: doi: 10.1177/01655515221093029
Zusammenfassung

Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter and rank the large and dynamic amount of information available on the Internet. Among several methodologies to perform such audits, virtual agents stand out because they offer the ability to perform systematic experiments, simulating human behaviour without the associated costs of recruiting participants. Motivated by the importance of research transparency and replicability of results, this article focuses on the challenges of such an approach. It provides methodological details, recommendations, lessons learned and limitations based on our experience of setting up experiments for eight search engines (including main, news, image and video sections) with hundreds of virtual agents placed in different regions. We demonstrate the successful performance of our research infrastructure across multiple data collections, with diverse experimental designs, and point to different changes and strategies that improve the quality of the method. We conclude that virtual agents are a promising venue for monitoring the performance of algorithms across long periods of time, and we hope that this article can serve as a basis for further research in this area.

Zusammenfassung in einer weiteren Sprache
Fachgebiet (DDC)
320 Politik
Schlagwörter
Algorithm auditing, data collection, search engine audits, user modelling
Konferenz
Rezension
undefined / . - undefined, undefined
Zitieren
ISO 690ULLOA, Roberto, Mykola MAKHORTYKH, Aleksandra URMAN, 2024. Scaling up search engine audits : Practical insights for algorithm auditing. In: Journal of Information Science. Sage. 2024, 50(2), pp. 404-419. ISSN 0165-5515. eISSN 1741-6485. Available under: doi: 10.1177/01655515221093029
BibTex
@article{Ulloa2024-04Scali-67821,
  year={2024},
  doi={10.1177/01655515221093029},
  title={Scaling up search engine audits : Practical insights for algorithm auditing},
  number={2},
  volume={50},
  issn={0165-5515},
  journal={Journal of Information Science},
  pages={404--419},
  author={Ulloa, Roberto and Makhortykh, Mykola and Urman, Aleksandra}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/67821">
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-09-19T07:57:35Z</dc:date>
    <dc:language>eng</dc:language>
    <dc:creator>Makhortykh, Mykola</dc:creator>
    <dc:rights>Attribution 4.0 International</dc:rights>
    <dcterms:title>Scaling up search engine audits : Practical insights for algorithm auditing</dcterms:title>
    <dc:contributor>Makhortykh, Mykola</dc:contributor>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dcterms:abstract>Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter and rank the large and dynamic amount of information available on the Internet. Among several methodologies to perform such audits, virtual agents stand out because they offer the ability to perform systematic experiments, simulating human behaviour without the associated costs of recruiting participants. Motivated by the importance of research transparency and replicability of results, this article focuses on the challenges of such an approach. It provides methodological details, recommendations, lessons learned and limitations based on our experience of setting up experiments for eight search engines (including main, news, image and video sections) with hundreds of virtual agents placed in different regions. We demonstrate the successful performance of our research infrastructure across multiple data collections, with diverse experimental designs, and point to different changes and strategies that improve the quality of the method. We conclude that virtual agents are a promising venue for monitoring the performance of algorithms across long periods of time, and we hope that this article can serve as a basis for further research in this area.</dcterms:abstract>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-09-19T07:57:35Z</dcterms:available>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/>
    <dc:creator>Ulloa, Roberto</dc:creator>
    <dc:creator>Urman, Aleksandra</dc:creator>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:contributor>Urman, Aleksandra</dc:contributor>
    <dcterms:issued>2024-04</dcterms:issued>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/67821"/>
    <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by/4.0/"/>
    <dc:contributor>Ulloa, Roberto</dc:contributor>
  </rdf:Description>
</rdf:RDF>
Interner Vermerk
xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter
Kontakt
URL der Originalveröffentl.
Prüfdatum der URL
Prüfungsdatum der Dissertation
Finanzierungsart
Kommentar zur Publikation
Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Ja
Diese Publikation teilen