Counteracting estimation bias and social influence to improve the wisdom of crowds

Thumbnail Image
Date
2018
Authors
Kao, Albert B.
Berdahl, Andrew M.
Hartnett, Andrew T.
Bak-Coleman, Joseph B.
Ioannou, Christos C.
Giam, Xingli
Editors
Contact
Journal ISSN
Electronic ISSN
ISBN
Bibliographical data
Publisher
Series
DOI (citable link)
ArXiv-ID
International patent number
Link to the license
EU project number
Project
Open Access publication
Collections
Restricted until
Title in another language
Research Projects
Organizational Units
Journal Issue
Publication type
Journal article
Publication status
Published
Published in
Interface : Journal of the Royal Society ; 15 (2018), 141. - 20180130. - ISSN 1742-5689. - eISSN 1742-5662
Abstract
Aggregating multiple non-expert opinions into a collective estimate can improve accuracy across many contexts. However, two sources of error can diminish collective wisdom: individual estimation biases and information sharing between individuals. Here, we measure individual biases and social influence rules in multiple experiments involving hundreds of individuals performing a classic numerosity estimation task. We first investigate how existing aggregation methods, such as calculating the arithmetic mean or the median, are influenced by these sources of error. We show that the mean tends to overestimate, and the median underestimate, the true value for a wide range of numerosities. Quantifying estimation bias, and mapping individual bias to collective bias, allows us to develop and validate three new aggregation measures that effectively counter sources of collective estimation error. In addition, we present results from a further experiment that quantifies the social influence rules that individuals employ when incorporating personal estimates with social information. We show that the corrected mean is remarkably robust to social influence, retaining high accuracy in the presence or absence of social influence, across numerosities and across different methods for averaging social information. Using knowledge of estimation biases and social influence rules may therefore be an inexpensive and general strategy to improve the wisdom of crowds.
Summary in another language
Subject (DDC)
570 Biosciences, Biology
Keywords
Conference
Review
undefined / . - undefined, undefined. - (undefined; undefined)
Cite This
ISO 690KAO, Albert B., Andrew M. BERDAHL, Andrew T. HARTNETT, Matthew J. LUTZ, Joseph B. BAK-COLEMAN, Christos C. IOANNOU, Xingli GIAM, Iain D. COUZIN, 2018. Counteracting estimation bias and social influence to improve the wisdom of crowds. In: Interface : Journal of the Royal Society. 15(141), 20180130. ISSN 1742-5689. eISSN 1742-5662. Available under: doi: 10.1098/rsif.2018.0130
BibTex
@article{Kao2018-04Count-42457,
  year={2018},
  doi={10.1098/rsif.2018.0130},
  title={Counteracting estimation bias and social influence to improve the wisdom of crowds},
  number={141},
  volume={15},
  issn={1742-5689},
  journal={Interface : Journal of the Royal Society},
  author={Kao, Albert B. and Berdahl, Andrew M. and Hartnett, Andrew T. and Lutz, Matthew J. and Bak-Coleman, Joseph B. and Ioannou, Christos C. and Giam, Xingli and Couzin, Iain D.},
  note={Article Number: 20180130}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/42457">
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <dc:contributor>Berdahl, Andrew M.</dc:contributor>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/28"/>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/42457/1/Kao_2-1btenuimit1eq5.pdf"/>
    <dcterms:title>Counteracting estimation bias and social influence to improve the wisdom of crowds</dcterms:title>
    <dc:contributor>Lutz, Matthew J.</dc:contributor>
    <dc:language>eng</dc:language>
    <dc:creator>Giam, Xingli</dc:creator>
    <dc:rights>terms-of-use</dc:rights>
    <dcterms:issued>2018-04</dcterms:issued>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/28"/>
    <dc:creator>Couzin, Iain D.</dc:creator>
    <dc:creator>Bak-Coleman, Joseph B.</dc:creator>
    <dc:creator>Lutz, Matthew J.</dc:creator>
    <dc:contributor>Kao, Albert B.</dc:contributor>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2018-05-30T07:07:42Z</dcterms:available>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2018-05-30T07:07:42Z</dc:date>
    <dc:contributor>Ioannou, Christos C.</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:creator>Hartnett, Andrew T.</dc:creator>
    <dc:contributor>Hartnett, Andrew T.</dc:contributor>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/42457/1/Kao_2-1btenuimit1eq5.pdf"/>
    <dc:creator>Ioannou, Christos C.</dc:creator>
    <dc:contributor>Bak-Coleman, Joseph B.</dc:contributor>
    <dcterms:abstract xml:lang="eng">Aggregating multiple non-expert opinions into a collective estimate can improve accuracy across many contexts. However, two sources of error can diminish collective wisdom: individual estimation biases and information sharing between individuals. Here, we measure individual biases and social influence rules in multiple experiments involving hundreds of individuals performing a classic numerosity estimation task. We first investigate how existing aggregation methods, such as calculating the arithmetic mean or the median, are influenced by these sources of error. We show that the mean tends to overestimate, and the median underestimate, the true value for a wide range of numerosities. Quantifying estimation bias, and mapping individual bias to collective bias, allows us to develop and validate three new aggregation measures that effectively counter sources of collective estimation error. In addition, we present results from a further experiment that quantifies the social influence rules that individuals employ when incorporating personal estimates with social information. We show that the corrected mean is remarkably robust to social influence, retaining high accuracy in the presence or absence of social influence, across numerosities and across different methods for averaging social information. Using knowledge of estimation biases and social influence rules may therefore be an inexpensive and general strategy to improve the wisdom of crowds.</dcterms:abstract>
    <dc:contributor>Couzin, Iain D.</dc:contributor>
    <dc:creator>Berdahl, Andrew M.</dc:creator>
    <dc:creator>Kao, Albert B.</dc:creator>
    <dc:contributor>Giam, Xingli</dc:contributor>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/42457"/>
  </rdf:Description>
</rdf:RDF>
Internal note
xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter
Contact
URL of original publication
Test date of URL
Examination date of dissertation
Method of financing
Comment on publication
Alliance license
Corresponding Authors der Uni Konstanz vorhanden
International Co-Authors
Bibliography of Konstanz
Yes
Refereed
Yes