Expertise screening in crowdsourcing image quality
| dc.contributor.author | Hosu, Vlad | |
| dc.contributor.author | Lin, Hanhe | |
| dc.contributor.author | Saupe, Dietmar | |
| dc.date.accessioned | 2019-01-21T10:49:24Z | |
| dc.date.available | 2019-01-21T10:49:24Z | |
| dc.date.issued | 2018 | eng |
| dc.description.abstract | We propose a screening approach to find reliable and effectively expert crowd workers in image quality assessment (IQA). Our method measures the users' ability to identify image degradations by using test questions, together with several relaxed reliability checks. We conduct multiple experiments, obtaining reproducible results with a high agreement between the expertise-screened crowd and the freelance experts of 0.95 Spearman rank order correlation (SROCC), with one restriction on the image type. Our contributions include a reliability screening method for uninformative users, a new type of test questions that rely on our proposed database 1 of pristine and artificially distorted images, a group agreement extrapolation method and an analysis of the crowdsourcing experiments. | eng |
| dc.description.version | published | eng |
| dc.identifier.doi | 10.1109/QoMEX.2018.8463427 | eng |
| dc.identifier.uri | https://kops.uni-konstanz.de/handle/123456789/44631 | |
| dc.language.iso | eng | eng |
| dc.subject.ddc | 004 | eng |
| dc.title | Expertise screening in crowdsourcing image quality | eng |
| dc.type | INPROCEEDINGS | eng |
| dspace.entity.type | Publication | |
| kops.citation.bibtex | @inproceedings{Hosu2018Exper-44631,
year={2018},
doi={10.1109/QoMEX.2018.8463427},
title={Expertise screening in crowdsourcing image quality},
isbn={978-1-5386-2605-4},
publisher={IEEE},
address={Piscataway, New Jersey, USA},
booktitle={2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX)},
pages={276--281},
author={Hosu, Vlad and Lin, Hanhe and Saupe, Dietmar}
} | |
| kops.citation.iso690 | HOSU, Vlad, Hanhe LIN, Dietmar SAUPE, 2018. Expertise screening in crowdsourcing image quality. 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). Cagliari, Italy, 29. Mai 2018 - 1. Juni 2018. In: 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). Piscataway, New Jersey, USA: IEEE, 2018, pp. 276-281. eISSN 2472-7814. ISBN 978-1-5386-2605-4. Available under: doi: 10.1109/QoMEX.2018.8463427 | deu |
| kops.citation.iso690 | HOSU, Vlad, Hanhe LIN, Dietmar SAUPE, 2018. Expertise screening in crowdsourcing image quality. 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). Cagliari, Italy, May 29, 2018 - Jun 1, 2018. In: 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). Piscataway, New Jersey, USA: IEEE, 2018, pp. 276-281. eISSN 2472-7814. ISBN 978-1-5386-2605-4. Available under: doi: 10.1109/QoMEX.2018.8463427 | eng |
| kops.citation.rdf | <rdf:RDF
xmlns:dcterms="http://purl.org/dc/terms/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:bibo="http://purl.org/ontology/bibo/"
xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
xmlns:foaf="http://xmlns.com/foaf/0.1/"
xmlns:void="http://rdfs.org/ns/void#"
xmlns:xsd="http://www.w3.org/2001/XMLSchema#" >
<rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/44631">
<dcterms:issued>2018</dcterms:issued>
<dc:creator>Hosu, Vlad</dc:creator>
<dc:creator>Saupe, Dietmar</dc:creator>
<void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
<dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2019-01-21T10:49:24Z</dc:date>
<dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2019-01-21T10:49:24Z</dcterms:available>
<dc:contributor>Hosu, Vlad</dc:contributor>
<dcterms:abstract xml:lang="eng">We propose a screening approach to find reliable and effectively expert crowd workers in image quality assessment (IQA). Our method measures the users' ability to identify image degradations by using test questions, together with several relaxed reliability checks. We conduct multiple experiments, obtaining reproducible results with a high agreement between the expertise-screened crowd and the freelance experts of 0.95 Spearman rank order correlation (SROCC), with one restriction on the image type. Our contributions include a reliability screening method for uninformative users, a new type of test questions that rely on our proposed database<sup> 1</sup> of pristine and artificially distorted images, a group agreement extrapolation method and an analysis of the crowdsourcing experiments.</dcterms:abstract>
<dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
<dcterms:title>Expertise screening in crowdsourcing image quality</dcterms:title>
<dc:language>eng</dc:language>
<bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/44631"/>
<dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
<dc:contributor>Saupe, Dietmar</dc:contributor>
<dc:creator>Lin, Hanhe</dc:creator>
<foaf:homepage rdf:resource="http://localhost:8080/"/>
<dc:contributor>Lin, Hanhe</dc:contributor>
</rdf:Description>
</rdf:RDF> | |
| kops.conferencefield | 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX), 29. Mai 2018 - 1. Juni 2018, Cagliari, Italy | deu |
| kops.date.conferenceEnd | 2018-06-01 | eng |
| kops.date.conferenceStart | 2018-05-29 | eng |
| kops.flag.knbibliography | true | |
| kops.location.conference | Cagliari, Italy | eng |
| kops.sourcefield | <i>2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX)</i>. Piscataway, New Jersey, USA: IEEE, 2018, pp. 276-281. eISSN 2472-7814. ISBN 978-1-5386-2605-4. Available under: doi: 10.1109/QoMEX.2018.8463427 | deu |
| kops.sourcefield.plain | 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). Piscataway, New Jersey, USA: IEEE, 2018, pp. 276-281. eISSN 2472-7814. ISBN 978-1-5386-2605-4. Available under: doi: 10.1109/QoMEX.2018.8463427 | deu |
| kops.sourcefield.plain | 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). Piscataway, New Jersey, USA: IEEE, 2018, pp. 276-281. eISSN 2472-7814. ISBN 978-1-5386-2605-4. Available under: doi: 10.1109/QoMEX.2018.8463427 | eng |
| kops.title.conference | 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX) | eng |
| relation.isAuthorOfPublication | 46e43f0d-5589-4060-b110-18519cbf61e0 | |
| relation.isAuthorOfPublication | 72057485-5f84-41aa-b6cb-8d616362e6a8 | |
| relation.isAuthorOfPublication | fffb576d-6ec6-4221-8401-77f1d117a9b9 | |
| relation.isAuthorOfPublication.latestForDiscovery | 46e43f0d-5589-4060-b110-18519cbf61e0 | |
| source.bibliographicInfo.fromPage | 276 | eng |
| source.bibliographicInfo.toPage | 281 | eng |
| source.identifier.eissn | 2472-7814 | eng |
| source.identifier.isbn | 978-1-5386-2605-4 | eng |
| source.publisher | IEEE | eng |
| source.publisher.location | Piscataway, New Jersey, USA | eng |
| source.title | 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX) | eng |