Critical analysis on the reproducibility of visual quality assessment using deep features

Lade...
Vorschaubild
Dateien
Goetz-Hahn_2-1slsft0xvpsz07.pdf
Goetz-Hahn_2-1slsft0xvpsz07.pdfGröße: 1.86 MBDownloads: 78
Datum
2022
Herausgeber:innen
Kontakt
ISSN der Zeitschrift
Electronic ISSN
ISBN
Bibliografische Daten
Verlag
Schriftenreihe
Auflagebezeichnung
ArXiv-ID
Internationale Patentnummer
Link zur Lizenz
Angaben zur Forschungsförderung
Projekt
Open Access-Veröffentlichung
Open Access Gold
Core Facility der Universität Konstanz
Gesperrt bis
Titel in einer weiteren Sprache
Forschungsvorhaben
Organisationseinheiten
Zeitschriftenheft
Publikationstyp
Zeitschriftenartikel
Publikationsstatus
Published
Erschienen in
Zusammenfassung

Data used to train supervised machine learning models are commonly split into independent training, validation, and test sets. This paper illustrates that complex data leakage cases have occurred in the no-reference image and video quality assessment literature. Recently, papers in several journals reported performance results well above the best in the field. However, our analysis shows that information from the test set was inappropriately used in the training process in different ways and that the claimed performance results cannot be achieved. When correcting for the data leakage, the performances of the approaches drop even below the state-of-the-art by a large margin. Additionally, we investigate end-to-end variations to the discussed approaches, which do not improve upon the original.

Zusammenfassung in einer weiteren Sprache
Fachgebiet (DDC)
004 Informatik
Schlagwörter
Konferenz
Rezension
undefined / . - undefined, undefined
Zitieren
ISO 690GÖTZ-HAHN, Franz, Vlad HOSU, Dietmar SAUPE, 2022. Critical analysis on the reproducibility of visual quality assessment using deep features. In: PLoS ONE. Public Library of Science (PLoS). 2022, 17(8), e0269715. eISSN 1932-6203. Available under: doi: 10.1371/journal.pone.0269715
BibTex
@article{GotzHahn2022Criti-59105,
  year={2022},
  doi={10.1371/journal.pone.0269715},
  title={Critical analysis on the reproducibility of visual quality assessment using deep features},
  number={8},
  volume={17},
  journal={PLoS ONE},
  author={Götz-Hahn, Franz and Hosu, Vlad and Saupe, Dietmar},
  note={Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID 251654672 –TRR 161 (Project A05). Article Number: e0269715}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/59105">
    <dc:rights>Attribution 4.0 International</dc:rights>
    <dc:language>eng</dc:language>
    <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by/4.0/"/>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2022-11-10T13:22:25Z</dcterms:available>
    <dcterms:abstract xml:lang="eng">Data used to train supervised machine learning models are commonly split into independent training, validation, and test sets. This paper illustrates that complex data leakage cases have occurred in the no-reference image and video quality assessment literature. Recently, papers in several journals reported performance results well above the best in the field. However, our analysis shows that information from the test set was inappropriately used in the training process in different ways and that the claimed performance results cannot be achieved. When correcting for the data leakage, the performances of the approaches drop even below the state-of-the-art by a large margin. Additionally, we investigate end-to-end variations to the discussed approaches, which do not improve upon the original.</dcterms:abstract>
    <dc:creator>Hosu, Vlad</dc:creator>
    <dc:creator>Saupe, Dietmar</dc:creator>
    <dcterms:title>Critical analysis on the reproducibility of visual quality assessment using deep features</dcterms:title>
    <dc:creator>Götz-Hahn, Franz</dc:creator>
    <dc:contributor>Saupe, Dietmar</dc:contributor>
    <dc:contributor>Hosu, Vlad</dc:contributor>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/59105"/>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/59105/1/Goetz-Hahn_2-1slsft0xvpsz07.pdf"/>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:contributor>Götz-Hahn, Franz</dc:contributor>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2022-11-10T13:22:25Z</dc:date>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/59105/1/Goetz-Hahn_2-1slsft0xvpsz07.pdf"/>
    <dcterms:issued>2022</dcterms:issued>
  </rdf:Description>
</rdf:RDF>
Interner Vermerk
xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter
Kontakt
URL der Originalveröffentl.
Prüfdatum der URL
Prüfungsdatum der Dissertation
Finanzierungsart
Kommentar zur Publikation
Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID 251654672 –TRR 161 (Project A05).
Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Ja
Diese Publikation teilen