DeepShapeKit : accurate 4D shape reconstruction of swimming fish

Lade...
Vorschaubild
Dateien
Zu diesem Dokument gibt es keine Dateien.
Datum
2022
Herausgeber:innen
Kontakt
ISSN der Zeitschrift
Electronic ISSN
ISBN
Bibliografische Daten
Verlag
Schriftenreihe
Auflagebezeichnung
URI (zitierfähiger Link)
ArXiv-ID
Internationale Patentnummer
Angaben zur Forschungsförderung
Projekt
Open Access-Veröffentlichung
Core Facility der Universität Konstanz
Gesperrt bis
Titel in einer weiteren Sprache
Forschungsvorhaben
Organisationseinheiten
Zeitschriftenheft
Publikationstyp
Beitrag zu einem Konferenzband
Publikationsstatus
Published
Erschienen in
MA, Shugen, ed., Yasushi NAKAUCHI, ed. and others. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 23–27, 2022, Kyoto International Conference Center, Japan. Piscataway, NJ: IEEE, 2022, pp. 526-531. ISSN 2153-0858. eISSN 2153-0866. ISBN 978-1-66547-928-8. Available under: doi: 10.1109/IROS47612.2022.9982097
Zusammenfassung

In this paper, we present methods for capturing 4D body shapes of swimming fish with affordable small training datasets and textureless 2D videos. Automated capture of spatiotemporal animal movements and postures is revolutionizing the study of collective animal behavior. 4D (including 3D space + time) shape data from animals like schooling fish contains a rich array of social and non-social information that can be used to shed light on the fundamental mechanisms underlying collective behavior. However, unlike the large datasets used for 4D shape reconstructions of the human body, there are no large amounts of labeled training datasets for reconstructing fish bodies in 4D, due to the difficulty of underwater data collection. We created a template mesh model using 3D scan data from a real fish, then extracted silhouettes (segmentation masks) and key-points of the fish body using Mask R-CNN and DeepLabCut, respectively. Next, using the Adam optimizer, we optimized the 3D template mesh model for each frame by minimizing the difference between the projected 3D model and the detected silhouettes as well as the key-points. Finally, using an LSTM-based smoother, we generated accurate 4D shapes of schooling fish based on the 3D shapes over each frame. Our results show that the method is effective for 4D shape reconstructions of swimming fish, with greater fidelity than other state-of-the-art algorithms.

Zusammenfassung in einer weiteren Sprache
Fachgebiet (DDC)
004 Informatik
Schlagwörter
Konferenz
International Conference on Intelligent Robots and Systems (IROS 2022), 23. Okt. 2022 - 27. Okt. 2022, Kyoto, Japan
Rezension
undefined / . - undefined, undefined
Zitieren
ISO 690WU, Ruiheng, Oliver DEUSSEN, Liang LI, 2022. DeepShapeKit : accurate 4D shape reconstruction of swimming fish. International Conference on Intelligent Robots and Systems (IROS 2022). Kyoto, Japan, 23. Okt. 2022 - 27. Okt. 2022. In: MA, Shugen, ed., Yasushi NAKAUCHI, ed. and others. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 23–27, 2022, Kyoto International Conference Center, Japan. Piscataway, NJ: IEEE, 2022, pp. 526-531. ISSN 2153-0858. eISSN 2153-0866. ISBN 978-1-66547-928-8. Available under: doi: 10.1109/IROS47612.2022.9982097
BibTex
@inproceedings{Wu2022DeepS-59666,
  year={2022},
  doi={10.1109/IROS47612.2022.9982097},
  title={DeepShapeKit : accurate 4D shape reconstruction of swimming fish},
  isbn={978-1-66547-928-8},
  issn={2153-0858},
  publisher={IEEE},
  address={Piscataway, NJ},
  booktitle={2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 23–27, 2022, Kyoto International Conference Center, Japan},
  pages={526--531},
  editor={Ma, Shugen and Nakauchi, Yasushi},
  author={Wu, Ruiheng and Deussen, Oliver and Li, Liang}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/59666">
    <dc:contributor>Wu, Ruiheng</dc:contributor>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:contributor>Deussen, Oliver</dc:contributor>
    <dc:creator>Li, Liang</dc:creator>
    <dc:language>eng</dc:language>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-01-10T13:31:05Z</dcterms:available>
    <dc:rights>terms-of-use</dc:rights>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/43615"/>
    <dc:contributor>Li, Liang</dc:contributor>
    <dcterms:rights rdf:resource="https://rightsstatements.org/page/InC/1.0/"/>
    <dc:creator>Wu, Ruiheng</dc:creator>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-01-10T13:31:05Z</dc:date>
    <dc:creator>Deussen, Oliver</dc:creator>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/59666"/>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dcterms:abstract xml:lang="eng">In this paper, we present methods for capturing 4D body shapes of swimming fish with affordable small training datasets and textureless 2D videos. Automated capture of spatiotemporal animal movements and postures is revolutionizing the study of collective animal behavior. 4D (including 3D space + time) shape data from animals like schooling fish contains a rich array of social and non-social information that can be used to shed light on the fundamental mechanisms underlying collective behavior. However, unlike the large datasets used for 4D shape reconstructions of the human body, there are no large amounts of labeled training datasets for reconstructing fish bodies in 4D, due to the difficulty of underwater data collection. We created a template mesh model using 3D scan data from a real fish, then extracted silhouettes (segmentation masks) and key-points of the fish body using Mask R-CNN and DeepLabCut, respectively. Next, using the Adam optimizer, we optimized the 3D template mesh model for each frame by minimizing the difference between the projected 3D model and the detected silhouettes as well as the key-points. Finally, using an LSTM-based smoother, we generated accurate 4D shapes of schooling fish based on the 3D shapes over each frame. Our results show that the method is effective for 4D shape reconstructions of swimming fish, with greater fidelity than other state-of-the-art algorithms.</dcterms:abstract>
    <dcterms:issued>2022</dcterms:issued>
    <dcterms:title>DeepShapeKit : accurate 4D shape reconstruction of swimming fish</dcterms:title>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/43615"/>
  </rdf:Description>
</rdf:RDF>
Interner Vermerk
xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter
Kontakt
URL der Originalveröffentl.
Prüfdatum der URL
Prüfungsdatum der Dissertation
Finanzierungsart
Kommentar zur Publikation
Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Diese Publikation teilen