Publikation:

VISITOR : Visual Interactive State Sequence Exploration for Reinforcement Learning

Lade...
Vorschaubild

Dateien

Metz_2-1b7letggi23f48.pdf
Metz_2-1b7letggi23f48.pdfGröße: 3.63 MBDownloads: 54

Datum

2023

Herausgeber:innen

Kontakt

ISSN der Zeitschrift

Electronic ISSN

ISBN

Bibliografische Daten

Verlag

Schriftenreihe

Auflagebezeichnung

DOI (zitierfähiger Link)
ArXiv-ID

Internationale Patentnummer

Link zur Lizenz

Angaben zur Forschungsförderung

Projekt

Open Access-Veröffentlichung
Open Access Hybrid
Core Facility der Universität Konstanz

Gesperrt bis

Titel in einer weiteren Sprache

Publikationstyp
Zeitschriftenartikel
Publikationsstatus
Published

Erschienen in

Computer Graphics Forum. Wiley. 2023, 42(3), S. 397-408. ISSN 0167-7055. eISSN 1467-8659. Verfügbar unter: doi: 10.1111/cgf.14839

Zusammenfassung

Understanding the behavior of deep reinforcement learning agents is a crucial requirement throughout their development. Existing work has addressed the identification of observable behavioral patterns in state sequences or analysis of isolated internal representations; however, the overall decision-making of deep-learning RL agents remains opaque. To tackle this, we present VISITOR, a visual analytics system enabling the analysis of entire state sequences, the diagnosis of singular predictions, and the comparison between agents. A sequence embedding view enables the multiscale analysis of state sequences, utilizing custom embedding techniques for a stable spatialization of the observations and internal states. We provide multiple layers: (1) a state space embedding, highlighting different groups of states inside the state-action sequences, (2) a trajectory view, emphasizing decision points, (3) a network activation mapping, visualizing the relationship between observations and network activations, (4) a transition embedding, enabling the analysis of state-to-state transitions. The embedding view is accompanied by an interactive reward view that captures the temporal development of metrics, which can be linked directly to states in the embedding. Lastly, a model list allows for the quick comparison of models across multiple metrics. Annotations can be exported to communicate results to different audiences. Our two-stage evaluation with eight experts confirms the effectiveness in identifying states of interest, comparing the quality of policies, and reasoning about the internal decision-making processes.

Zusammenfassung in einer weiteren Sprache

Fachgebiet (DDC)
004 Informatik

Schlagwörter

Konferenz

Rezension
undefined / . - undefined, undefined

Forschungsvorhaben

Organisationseinheiten

Zeitschriftenheft

Zugehörige Datensätze in KOPS

Zitieren

ISO 690METZ, Yannick, Eugene BYKOVETS, Lucas JOOS, Daniel A. KEIM, Mennatallah EL-ASSADY, 2023. VISITOR : Visual Interactive State Sequence Exploration for Reinforcement Learning. In: Computer Graphics Forum. Wiley. 2023, 42(3), S. 397-408. ISSN 0167-7055. eISSN 1467-8659. Verfügbar unter: doi: 10.1111/cgf.14839
BibTex
@article{Metz2023-06VISIT-67521,
  year={2023},
  doi={10.1111/cgf.14839},
  title={VISITOR : Visual Interactive State Sequence Exploration for Reinforcement Learning},
  number={3},
  volume={42},
  issn={0167-7055},
  journal={Computer Graphics Forum},
  pages={397--408},
  author={Metz, Yannick and Bykovets, Eugene and Joos, Lucas and Keim, Daniel A. and El-Assady, Mennatallah}
}
RDF
<rdf:RDF
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:bibo="http://purl.org/ontology/bibo/"
    xmlns:dspace="http://digital-repositories.org/ontologies/dspace/0.1.0#"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:void="http://rdfs.org/ns/void#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#" > 
  <rdf:Description rdf:about="https://kops.uni-konstanz.de/server/rdf/resource/123456789/67521">
    <dc:creator>Joos, Lucas</dc:creator>
    <dc:contributor>El-Assady, Mennatallah</dc:contributor>
    <dcterms:issued>2023-06</dcterms:issued>
    <dc:contributor>Metz, Yannick</dc:contributor>
    <dspace:isPartOfCollection rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dcterms:title>VISITOR : Visual Interactive State Sequence Exploration for Reinforcement Learning</dcterms:title>
    <dc:contributor>Keim, Daniel A.</dc:contributor>
    <bibo:uri rdf:resource="https://kops.uni-konstanz.de/handle/123456789/67521"/>
    <dcterms:available rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-08-07T14:53:20Z</dcterms:available>
    <dc:creator>Keim, Daniel A.</dc:creator>
    <dcterms:hasPart rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/67521/1/Metz_2-1b7letggi23f48.pdf"/>
    <dcterms:abstract>Understanding the behavior of deep reinforcement learning agents is a crucial requirement throughout their development. Existing work has addressed the identification of observable behavioral patterns in state sequences or analysis of isolated internal representations; however, the overall decision-making of deep-learning RL agents remains opaque. To tackle this, we present VISITOR, a visual analytics system enabling the analysis of entire state sequences, the diagnosis of singular predictions, and the comparison between agents. A sequence embedding view enables the multiscale analysis of state sequences, utilizing custom embedding techniques for a stable spatialization of the observations and internal states. We provide multiple layers: (1) a state space embedding, highlighting different groups of states inside the state-action sequences, (2) a trajectory view, emphasizing decision points, (3) a network activation mapping, visualizing the relationship between observations and network activations, (4) a transition embedding, enabling the analysis of state-to-state transitions. The embedding view is accompanied by an interactive reward view that captures the temporal development of metrics, which can be linked directly to states in the embedding. Lastly, a model list allows for the quick comparison of models across multiple metrics. Annotations can be exported to communicate results to different audiences. Our two-stage evaluation with eight experts confirms the effectiveness in identifying states of interest, comparing the quality of policies, and reasoning about the internal decision-making processes.</dcterms:abstract>
    <dc:contributor>Joos, Lucas</dc:contributor>
    <dc:date rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-08-07T14:53:20Z</dc:date>
    <dcterms:rights rdf:resource="http://creativecommons.org/licenses/by-nc/4.0/"/>
    <foaf:homepage rdf:resource="http://localhost:8080/"/>
    <dspace:hasBitstream rdf:resource="https://kops.uni-konstanz.de/bitstream/123456789/67521/1/Metz_2-1b7letggi23f48.pdf"/>
    <void:sparqlEndpoint rdf:resource="http://localhost/fuseki/dspace/sparql"/>
    <dc:creator>Bykovets, Eugene</dc:creator>
    <dc:rights>Attribution-NonCommercial 4.0 International</dc:rights>
    <dc:contributor>Bykovets, Eugene</dc:contributor>
    <dc:creator>El-Assady, Mennatallah</dc:creator>
    <dc:creator>Metz, Yannick</dc:creator>
    <dcterms:isPartOf rdf:resource="https://kops.uni-konstanz.de/server/rdf/resource/123456789/36"/>
    <dc:language>eng</dc:language>
  </rdf:Description>
</rdf:RDF>

Interner Vermerk

xmlui.Submission.submit.DescribeStep.inputForms.label.kops_note_fromSubmitter

Kontakt
URL der Originalveröffentl.

Prüfdatum der URL

Prüfungsdatum der Dissertation

Finanzierungsart

Kommentar zur Publikation

Allianzlizenz
Corresponding Authors der Uni Konstanz vorhanden
Internationale Co-Autor:innen
Universitätsbibliographie
Ja
Begutachtet
Ja
Diese Publikation teilen