Interactive Systems
Machine Learning
Interactive Systems
Spoken Dialogue Systems Optimization by Reinforcement Learning
> MALIS Home > Research Topics > Interactive Systems > Spoken Dialogue Systems Optimization by Reinforcement Learning

MAchine Learning and Interactive Systems

Spoken Dialogue Systems Optimization by Reinforcement Learning
  by Pietquin Olivier

During the last decade, research in the field of Spoken Dialogue
Systems (SDS) has experienced increasing growth. However,
the design and optimization of SDS is not only about combining
speech and language processing systems such as Automatic
Speech Recognition (ASR), parsers, Natural Language Generation
(NLG), and Text-to-Speech (TTS) synthesis systems. It
also requires the development of dialogue strategies taking at
least into account the performances of these subsystems (and
others), the nature of the task (e.g. form filling, tutoring, robot
control, or database search/browsing), and the user’s behaviour
(e.g. cooperativeness, expertise). Due to the great variability
of these factors, reuse of previous hand-crafted designs is also
made very difficult. For these reasons, statistical machine learning
(ML) methods applied to automatic SDS optimization have
been a leading research area for the last few years. Among machine learning methods, Reinforcement Learning (RL) has proven to be quite effective at optimizing dialogue strategies because of its ability to learn from rewarded interactions. Unlike supervised learning, RL doesn’t require any sample of perfect interaction which is anyway not accessible in the case of human-machine interaction. Indeed, optimality is not easy to define in the field of man-machine dialogue while it is easier to obtain a rating from a user after s/he used the system. This is why reinforcement learning for spoken dialogue systems optimization is studied since the mid 90’s.


Some publications on the topic

Book : O. PIETQUIN, "A Framework for Unsupervised Learning of Dialogue Strategies", Presses Universitaires de Louvain, SIMILAR Collection, 246 pages, 2004.

The European CLASSiC project