Publikation: Multi-Object Tracking and Pose Estimation for Animals
Dateien
Datum
Autor:innen
Herausgeber:innen
ISSN der Zeitschrift
Electronic ISSN
ISBN
Bibliografische Daten
Verlag
Schriftenreihe
Auflagebezeichnung
URI (zitierfähiger Link)
Internationale Patentnummer
Link zur Lizenz
Angaben zur Forschungsförderung
Projekt
Open Access-Veröffentlichung
Core Facility der Universität Konstanz
Titel in einer weiteren Sprache
Publikationstyp
Publikationsstatus
Erschienen in
Zusammenfassung
The study of collective behaviour among animals hinges on accurately estimating and tracking their movements. Central to our research at the Centre for the Advanced Study of Collective Behaviour is the challenge of accurately estimating the poses and tracking the movements of multiple animals in their natural habitats.
One of the primary challenges in studying animal collective behaviour is the scarcity of data, which predominantly focuses on human subjects, with a notable deficiency in annotated data for animals. Although there are some existing animal datasets, they often encompass various species, limiting their utility for specific research purposes. Moreover, the availability of annotated multi-instance animal data remains sparse compared to human datasets, further exacerbating the resource gap.
We propose novel approaches to address this disparity and facilitate advancements in pose estimation and tracking methodologies for collective behaviour studies. Firstly, we leverage pigeon data collected in controllable indoor environments to train models capable of performing reliably in wild settings. We also demonstrate that it is possible to train a model on data containing a single pigeon to predict keypoints from multiple pigeons stably and accurately. This provides an alternative in the domain shift to other species. With interactive speed, this model tracks and estimates the 3D poses of up to ten pigeons. Second, we explore unsupervised label propagation that obviates the need for annotated data to propagate poses through video sequences. Our pipeline can effectively track the posture of small objects relative to the frame size, enhancing the applicability. Third, our pioneering 3D pose estimation pipeline, trained exclusively on synthetic data, robustly predicts keypoints from multi-view silhouettes and is thus robust to transformations that leave silhouettes unchanged, such as variations in texture and lighting. This method successfully narrows the domain gap where real-world annotations are scarce by leveraging synthetic data. Lastly, to the best of our knowledge, we are the first to offer a pipeline for neural rendering of textures, facilitating downstream tasks such as individual re-identification. Our method offers an efficient alternative to existing approaches based on convolutional neural networks (CNNs) and vision transformers, operating at interactive speeds.
We think that our contributions promote systematic advancements in the study of animal collective behaviour and offer novel methodologies for 3D pose estimation and individual re-identification.
Zusammenfassung in einer weiteren Sprache
Fachgebiet (DDC)
Schlagwörter
Konferenz
Rezension
Zitieren
ISO 690
WALDMANN, Urs, 2024. Multi-Object Tracking and Pose Estimation for Animals [Dissertation]. Konstanz: Universität KonstanzBibTex
@phdthesis{Waldmann2024Multi-71609, year={2024}, title={Multi-Object Tracking and Pose Estimation for Animals}, author={Waldmann, Urs}, address={Konstanz}, school={Universität Konstanz} }
RDF
<rdf:RDF xmlns:dcterms="http://purl.org/dc/terms/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:bibo="http://purl.org/ontology/bibo/" xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:void="http://rdfs.org/ns/void#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/71609"> <dc:creator>Waldmann, Urs</dc:creator> <dc:contributor>Waldmann, Urs</dc:contributor> <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/> <foaf:homepage rdf:resource="http://localhost:8080/"/> <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by/4.0/"/> <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/43615"/> <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/71609/4/Waldmann_2-g0pqp1tx0xa40.pdf"/> <dc:language>eng</dc:language> <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2024-12-09T10:08:39Z</dcterms:available> <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/71609/4/Waldmann_2-g0pqp1tx0xa40.pdf"/> <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2024-12-09T10:08:39Z</dc:date> <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/43615"/> <dcterms:title>Multi-Object Tracking and Pose Estimation for Animals</dcterms:title> <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/71609"/> <dc:rights>Attribution 4.0 International</dc:rights> <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/> <dcterms:abstract>The study of collective behaviour among animals hinges on accurately estimating and tracking their movements. Central to our research at the Centre for the Advanced Study of Collective Behaviour is the challenge of accurately estimating the poses and tracking the movements of multiple animals in their natural habitats. One of the primary challenges in studying animal collective behaviour is the scarcity of data, which predominantly focuses on human subjects, with a notable deficiency in annotated data for animals. Although there are some existing animal datasets, they often encompass various species, limiting their utility for specific research purposes. Moreover, the availability of annotated multi-instance animal data remains sparse compared to human datasets, further exacerbating the resource gap. We propose novel approaches to address this disparity and facilitate advancements in pose estimation and tracking methodologies for collective behaviour studies. Firstly, we leverage pigeon data collected in controllable indoor environments to train models capable of performing reliably in wild settings. We also demonstrate that it is possible to train a model on data containing a single pigeon to predict keypoints from multiple pigeons stably and accurately. This provides an alternative in the domain shift to other species. With interactive speed, this model tracks and estimates the 3D poses of up to ten pigeons. Second, we explore unsupervised label propagation that obviates the need for annotated data to propagate poses through video sequences. Our pipeline can effectively track the posture of small objects relative to the frame size, enhancing the applicability. Third, our pioneering 3D pose estimation pipeline, trained exclusively on synthetic data, robustly predicts keypoints from multi-view silhouettes and is thus robust to transformations that leave silhouettes unchanged, such as variations in texture and lighting. This method successfully narrows the domain gap where real-world annotations are scarce by leveraging synthetic data. Lastly, to the best of our knowledge, we are the first to offer a pipeline for neural rendering of textures, facilitating downstream tasks such as individual re-identification. Our method offers an efficient alternative to existing approaches based on convolutional neural networks (CNNs) and vision transformers, operating at interactive speeds. We think that our contributions promote systematic advancements in the study of animal collective behaviour and offer novel methodologies for 3D pose estimation and individual re-identification.</dcterms:abstract> <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/> <dcterms:issued>2024</dcterms:issued> </rdf:Description> </rdf:RDF>