Type of Publication: | Contribution to a conference |
Author: | Scheffler, Tatjana; Roller, Roland; Reithinger, Norbert |
Year of publication: | 2011 |
Published in: | Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop / Delgado, Ramón López-Cózar; Kobayashi, Tetsunori (ed.). - New York, NY : Springer New York, 2011. - pp. 295-300. - ISBN 978-1-4614-1334-9 |
DOI (citable link): | https://dx.doi.org/10.1007/978-1-4614-1335-6_29 |
Summary: |
In this paper, we present the SpeechEval user simulation platform for spoken dialog system (SDS) evaluation. Our user simulation realizes a complete end-to-end user test system with spoken input and output that calls speech-based dialog systems via telephone, and is intended to be used in the development and evaluation of real-life SDSs. We give an overview of the system architecture and present the results of applying our approach to two different kinds of systems in two domains.
User simulations for SDS are developed for many different purposes, often to test or train specific system components, like dialog manager, language model or the speech recognition itself. Many user simulations are developed in the context of research systems, which differ from commercially used systems in many aspects. In contrast, our user simulation uses a speech interface, can be carried over to new domains easily by learning characteristics of user behavior from a corpus, and treats the evaluated SDS as a black box. |
Subject (DDC): | 400 Philology, Linguistics |
Files | Size | Format | View |
---|---|---|---|
There are no files associated with this item. |
SCHEFFLER, Tatjana, Roland ROLLER, Norbert REITHINGER, 2011. SpeechEval : A Domain-Independent User Simulation Platform for Spoken Dialog System Evaluation. In: DELGADO, Ramón López-Cózar, ed., Tetsunori KOBAYASHI, ed.. Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. New York, NY:Springer New York, pp. 295-300. ISBN 978-1-4614-1334-9. Available under: doi: 10.1007/978-1-4614-1335-6_29
@inproceedings{Scheffler2011-08-12Speec-29024, title={SpeechEval : A Domain-Independent User Simulation Platform for Spoken Dialog System Evaluation}, year={2011}, doi={10.1007/978-1-4614-1335-6_29}, isbn={978-1-4614-1334-9}, address={New York, NY}, publisher={Springer New York}, booktitle={Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop}, pages={295--300}, editor={Delgado, Ramón López-Cózar and Kobayashi, Tetsunori}, author={Scheffler, Tatjana and Roller, Roland and Reithinger, Norbert} }