KonVid-150k : A Dataset for No-Reference Video Quality Assessment of Videos in-the-Wild
KonVid-150k : A Dataset for No-Reference Video Quality Assessment of Videos in-the-Wild
Loading...
Date
2021
Editors
Journal ISSN
Electronic ISSN
ISBN
Bibliographical data
Publisher
Series
URI (citable link)
DOI (citable link)
International patent number
Link to the license
EU project number
Project
Open Access publication
Collections
Title in another language
Publication type
Journal article
Publication status
Published
Published in
IEEE Access ; 9 (2021). - pp. 72139-72160. - IEEE. - eISSN 2169-3536
Abstract
Video quality assessment (VQA) methods focus on particular degradation types, usually artificially induced on a small set of reference videos. Hence, most traditional VQA methods under-perform in-the-wild. Deep learning approaches have had limited success due to the small size and diversity of existing VQA datasets, either artificial or authentically distorted. We introduce a new in-the-wild VQA dataset that is substantially larger and diverse: KonVid-150k. It consists of a coarsely annotated set of 153,841 videos having five quality ratings each, and 1,596 videos with a minimum of 89 ratings each. Additionally, we propose new efficient VQA approaches (MLSP-VQA) relying on multi-level spatially pooled deep-features (MLSP). They are exceptionally well suited for training at scale, compared to deep transfer learning approaches. Our best method, MLSP-VQA-FF, improves the Spearman rank-order correlation coefficient (SRCC) performance metric on the commonly used KoNViD-1k in-the-wild benchmark dataset to 0.82. It surpasses the best existing deep-learning model (0.80 SRCC) and hand-crafted feature-based method (0.78 SRCC). We further investigate how alternative approaches perform under different levels of label noise, and dataset size, showing that MLSP-VQA-FF is the overall best method for videos in-the-wild. Finally, we show that the MLSP-VQA models trained on KonVid-150k sets the new state-of-the-art for cross-test performance on KoNViD-1k and LIVE-Qualcomm with a 0.83 and 0.64 SRCC, respectively. For KoNViD-1k this inter-dataset testing outperforms intra-dataset experiments, showing excellent generalization.
Summary in another language
Subject (DDC)
004 Computer Science
Keywords
Datasets, deep transfer learning, multi-level spatially-pooled features, video quality assessment, video quality dataset
Conference
Review
undefined / . - undefined, undefined. - (undefined; undefined)
Cite This
ISO 690
GÖTZ-HAHN, Franz, Vlad HOSU, Hanhe LIN, Dietmar SAUPE, 2021. KonVid-150k : A Dataset for No-Reference Video Quality Assessment of Videos in-the-Wild. In: IEEE Access. IEEE. 9, pp. 72139-72160. eISSN 2169-3536. Available under: doi: 10.1109/ACCESS.2021.3077642BibTex
@article{GotzHahn2021KonVi-53766, year={2021}, doi={10.1109/ACCESS.2021.3077642}, title={KonVid-150k : A Dataset for No-Reference Video Quality Assessment of Videos in-the-Wild}, volume={9}, journal={IEEE Access}, pages={72139--72160}, author={Götz-Hahn, Franz and Hosu, Vlad and Lin, Hanhe and Saupe, Dietmar} }
RDF
<rdf:RDF xmlns:dcterms="http://purl.org/dc/terms/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:bibo="http://purl.org/ontology/bibo/" xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:void="http://rdfs.org/ns/void#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/53766"> <dcterms:issued>2021</dcterms:issued> <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/> <dc:language>eng</dc:language> <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/53766/1/Goetz-Hahn_2-1t0bmybzek49b5.pdf"/> <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/53766"/> <dc:creator>Saupe, Dietmar</dc:creator> <dc:contributor>Lin, Hanhe</dc:contributor> <dc:creator>Hosu, Vlad</dc:creator> <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/53766/1/Goetz-Hahn_2-1t0bmybzek49b5.pdf"/> <dcterms:abstract xml:lang="eng">Video quality assessment (VQA) methods focus on particular degradation types, usually artificially induced on a small set of reference videos. Hence, most traditional VQA methods under-perform in-the-wild. Deep learning approaches have had limited success due to the small size and diversity of existing VQA datasets, either artificial or authentically distorted. We introduce a new in-the-wild VQA dataset that is substantially larger and diverse: KonVid-150k. It consists of a coarsely annotated set of 153,841 videos having five quality ratings each, and 1,596 videos with a minimum of 89 ratings each. Additionally, we propose new efficient VQA approaches (MLSP-VQA) relying on multi-level spatially pooled deep-features (MLSP). They are exceptionally well suited for training at scale, compared to deep transfer learning approaches. Our best method, MLSP-VQA-FF, improves the Spearman rank-order correlation coefficient (SRCC) performance metric on the commonly used KoNViD-1k in-the-wild benchmark dataset to 0.82. It surpasses the best existing deep-learning model (0.80 SRCC) and hand-crafted feature-based method (0.78 SRCC). We further investigate how alternative approaches perform under different levels of label noise, and dataset size, showing that MLSP-VQA-FF is the overall best method for videos in-the-wild. Finally, we show that the MLSP-VQA models trained on KonVid-150k sets the new state-of-the-art for cross-test performance on KoNViD-1k and LIVE-Qualcomm with a 0.83 and 0.64 SRCC, respectively. For KoNViD-1k this inter-dataset testing outperforms intra-dataset experiments, showing excellent generalization.</dcterms:abstract> <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by/4.0/"/> <dcterms:title>KonVid-150k : A Dataset for No-Reference Video Quality Assessment of Videos in-the-Wild</dcterms:title> <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/> <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-05-27T09:36:18Z</dcterms:available> <dc:creator>Götz-Hahn, Franz</dc:creator> <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/> <dc:contributor>Hosu, Vlad</dc:contributor> <dc:contributor>Götz-Hahn, Franz</dc:contributor> <dc:rights>Attribution 4.0 International</dc:rights> <dc:contributor>Saupe, Dietmar</dc:contributor> <foaf:homepage rdf:resource="http://localhost:8080/"/> <dc:creator>Lin, Hanhe</dc:creator> <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2021-05-27T09:36:18Z</dc:date> </rdf:Description> </rdf:RDF>
Internal note
xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter
Examination date of dissertation
Method of financing
Comment on publication
Alliance license
Corresponding Authors der Uni Konstanz vorhanden
International Co-Authors
Bibliography of Konstanz
Yes
Refereed
Yes