KOPS - The Institutional Repository of the University of Konstanz

SpeechEval : A Domain-Independent User Simulation Platform for Spoken Dialog System Evaluation

SpeechEval : A Domain-Independent User Simulation Platform for Spoken Dialog System Evaluation

Type of Publication: Contribution to a conference
Author: Scheffler, Tatjana; Roller, Roland; Reithinger, Norbert
Year of publication: 2011
Published in: Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop / Delgado, Ramón López-Cózar; Kobayashi, Tetsunori (ed.). - New York, NY : Springer New York, 2011. - pp. 295-300. - ISBN 978-1-4614-1334-9
DOI (citable link): https://dx.doi.org/10.1007/978-1-4614-1335-6_29
Summary:
In this paper, we present the SpeechEval user simulation platform for spoken dialog system (SDS) evaluation. Our user simulation realizes a complete end-to-end user test system with spoken input and output that calls speech-based dialog systems via telephone, and is intended to be used in the development and evaluation of real-life SDSs. We give an overview of the system architecture and present the results of applying our approach to two different kinds of systems in two domains.

User simulations for SDS are developed for many different purposes, often to test or train specific system components, like dialog manager, language model or the speech recognition itself. Many user simulations are developed in the context of research systems, which differ from commercially used systems in many aspects. In contrast, our user simulation uses a speech interface, can be carried over to new domains easily by learning characteristics of user behavior from a corpus, and treats the evaluated SDS as a black box.
Subject (DDC): 400 Philology, Linguistics

Cite This

Files in this item

Files Size Format View

There are no files associated with this item.

SCHEFFLER, Tatjana, Roland ROLLER, Norbert REITHINGER, 2011. SpeechEval : A Domain-Independent User Simulation Platform for Spoken Dialog System Evaluation. In: DELGADO, Ramón López-Cózar, ed., Tetsunori KOBAYASHI, ed.. Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. New York, NY:Springer New York, pp. 295-300. ISBN 978-1-4614-1334-9. Available under: doi: 10.1007/978-1-4614-1335-6_29

@inproceedings{Scheffler2011-08-12Speec-29024, title={SpeechEval : A Domain-Independent User Simulation Platform for Spoken Dialog System Evaluation}, year={2011}, doi={10.1007/978-1-4614-1335-6_29}, isbn={978-1-4614-1334-9}, address={New York, NY}, publisher={Springer New York}, booktitle={Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop}, pages={295--300}, editor={Delgado, Ramón López-Cózar and Kobayashi, Tetsunori}, author={Scheffler, Tatjana and Roller, Roland and Reithinger, Norbert} }

Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop / ed. by Ramón López-Cózar Delgado ... - New York, NY : Springer, 2011. - S. 295-300. - ISBN 978-1-4614-1334-9 Roller, Roland 2014-09-30T13:33:45Z terms-of-use Reithinger, Norbert 2014-09-30T13:33:45Z Roller, Roland Scheffler, Tatjana 2011-08-12 In this paper, we present the SpeechEval user simulation platform for spoken dialog system (SDS) evaluation. Our user simulation realizes a complete end-to-end user test system with spoken input and output that calls speech-based dialog systems via telephone, and is intended to be used in the development and evaluation of real-life SDSs. We give an overview of the system architecture and present the results of applying our approach to two different kinds of systems in two domains.<br /><br />User simulations for SDS are developed for many different purposes, often to test or train specific system components, like dialog manager, language model or the speech recognition itself. Many user simulations are developed in the context of research systems, which differ from commercially used systems in many aspects. In contrast, our user simulation uses a speech interface, can be carried over to new domains easily by learning characteristics of user behavior from a corpus, and treats the evaluated SDS as a black box. eng SpeechEval : A Domain-Independent User Simulation Platform for Spoken Dialog System Evaluation Reithinger, Norbert Scheffler, Tatjana

This item appears in the following Collection(s)

Search KOPS


Browse

My Account