Dissertation Proposal: Zhiwen Tang
A Zoom link for the proposal will be distributed through the department mailing lists, or can be obtained by contacting Zhiwen directly.
Title: “Deep Reinforcement Learning for Interactive Systems”
One goal artificial intelligence (AI) has is to build virtual agents that interact with and assist humans. During the interaction, the agent learns the requirements from and completes the task together with the human user. The interactive setting and goal-oriented objective make them natural applications of reinforcement learning (RL). Retrieval-based systems are a popular branch of the interactive agents, where the agent retrieves relevant answers from a document collection or knowledge base. Early RL solutions to retrieval-based systems build on top of non-differentiable retrieval functions optimized over top ranks. However, in such a formulation, the agent may lose the global picture of the task and cannot have direct control over the search results. Those factors lead to insufficient exploration. The exploration is further discouraged by the assumption that the user is always willing to cooperate, resulting in sub-optimal performance.
In this thesis, I formulate retrieval-based tasks as an RL problem and propose a general framework for training and evaluation. I propose to present the agent with a global representation of the knowledge repository to enable the exploration of the full state and action space. I then employ a differentiable retrieval function to allow the agent to control the retrieval process directly. As a testbed for interactive systems, I co-organized the Dynamic Domain Track (DD) at the Text REtrieval Conference (TREC), where we introduce a simulated user to enable live interactions and reproducible evaluations. I also improve the fairness of evaluation by normalizing the unbounded metrics. In future work, I will study how to improve the exploration and robustness by adding competitiveness into the human-agent interaction.