Publikation:

Scaling up search engine audits : Practical insights for algorithm auditing

Lade...
Vorschaubild

Dateien

Ulloa_2-9iiu7vwo72ex9.pdf
Ulloa_2-9iiu7vwo72ex9.pdfGröße: 1.89 MBDownloads: 5

Datum

2024

Autor:innen

Makhortykh, Mykola
Urman, Aleksandra

Herausgeber:innen

Kontakt

ISSN der Zeitschrift

Electronic ISSN

ISBN

Bibliografische Daten

Verlag

Schriftenreihe

Auflagebezeichnung

ArXiv-ID

Internationale Patentnummer

Link zur Lizenz

Angaben zur Forschungsförderung

Swiss National Science Foundation: 100001CL_182630/1

Projekt

Open Access-Veröffentlichung
Open Access Hybrid
Core Facility der Universität Konstanz

Gesperrt bis

Titel in einer weiteren Sprache

Publikationstyp
Zeitschriftenartikel
Publikationsstatus
Published

Erschienen in

Journal of Information Science. Sage. 2024, 50(2), S. 404-419. ISSN 0165-5515. eISSN 1741-6485. Verfügbar unter: doi: 10.1177/01655515221093029

Zusammenfassung

Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter and rank the large and dynamic amount of information available on the Internet. Among several methodologies to perform such audits, virtual agents stand out because they offer the ability to perform systematic experiments, simulating human behaviour without the associated costs of recruiting participants. Motivated by the importance of research transparency and replicability of results, this article focuses on the challenges of such an approach. It provides methodological details, recommendations, lessons learned and limitations based on our experience of setting up experiments for eight search engines (including main, news, image and video sections) with hundreds of virtual agents placed in different regions. We demonstrate the successful performance of our research infrastructure across multiple data collections, with diverse experimental designs, and point to different changes and strategies that improve the quality of the method. We conclude that virtual agents are a promising venue for monitoring the performance of algorithms across long periods of time, and we hope that this article can serve as a basis for further research in this area.

Zusammenfassung in einer weiteren Sprache

Fachgebiet (DDC)
320 Politik

Schlagwörter

Algorithm auditing, data collection, search engine audits, user modelling

Konferenz

Rezension
undefined / . - undefined, undefined

Forschungsvorhaben

Organisationseinheiten

Zeitschriftenheft

Zugehörige Datensätze in KOPS

Zitieren

ISO 690ULLOA, Roberto, Mykola MAKHORTYKH, Aleksandra URMAN, 2024. Scaling up search engine audits : Practical insights for algorithm auditing. In: Journal of Information Science. Sage. 2024, 50(2), S. 404-419. ISSN 0165-5515. eISSN 1741-6485. Verfügbar unter: doi: 10.1177/01655515221093029
BibTex
@article{Ulloa2024-04Scali-67821,
  year={2024},
  doi={10.1177/01655515221093029},
  title={Scaling up search engine audits : Practical insights for algorithm auditing},
  number={2},
  volume={50},
  issn={0165-5515},
  journal={Journal of Information Science},
  pages={404--419},
  author={Ulloa, Roberto and Makhortykh, Mykola and Urman, Aleksandra}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/67821">
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-09-19T07:57:35Z</dc:date>
    <dc:language>eng</dc:language>
    <dc:creator>Makhortykh, Mykola</dc:creator>
    <dc:rights>Attribution 4.0 International</dc:rights>
    <dcterms:title>Scaling up search engine audits : Practical insights for algorithm auditing</dcterms:title>
    <dc:contributor>Makhortykh, Mykola</dc:contributor>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dcterms:abstract>Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter and rank the large and dynamic amount of information available on the Internet. Among several methodologies to perform such audits, virtual agents stand out because they offer the ability to perform systematic experiments, simulating human behaviour without the associated costs of recruiting participants. Motivated by the importance of research transparency and replicability of results, this article focuses on the challenges of such an approach. It provides methodological details, recommendations, lessons learned and limitations based on our experience of setting up experiments for eight search engines (including main, news, image and video sections) with hundreds of virtual agents placed in different regions. We demonstrate the successful performance of our research infrastructure across multiple data collections, with diverse experimental designs, and point to different changes and strategies that improve the quality of the method. We conclude that virtual agents are a promising venue for monitoring the performance of algorithms across long periods of time, and we hope that this article can serve as a basis for further research in this area.</dcterms:abstract>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-09-19T07:57:35Z</dcterms:available>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42"/>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/67821/1/Ulloa_2-9iiu7vwo72ex9.pdf"/>
    <dc:creator>Ulloa, Roberto</dc:creator>
    <dc:creator>Urman, Aleksandra</dc:creator>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:contributor>Urman, Aleksandra</dc:contributor>
    <dcterms:issued>2024-04</dcterms:issued>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/67821"/>
    <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by/4.0/"/>
    <dc:contributor>Ulloa, Roberto</dc:contributor>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/67821/1/Ulloa_2-9iiu7vwo72ex9.pdf"/>
  </rdf:Description>
</rdf:RDF>

Interner Vermerk

xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter

Kontakt
URL der Originalveröffentl.

Prüfdatum der URL

Prüfungsdatum der Dissertation

Finanzierungsart

Kommentar zur Publikation

Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Ja
Diese Publikation teilen