PhD Dissertation Defense: Zhiwen Tang

A Zoom link for the defense will be distributed through the department mailing lists, or can be obtained by contacting Zhiwen directly.

Title: “Deep Reinforcement Learning for Interactive Systems”

One goal of artificial intelligence (AI) is to build intelligent systems that can interact with and assist humans. During the interaction, a system learns the requirements from the human user and adapts to the needs to complete tasks. Because of the interactive setting and the goal-oriented objective, reinforcement learning (RL) is a trending solution. Retrieval-based interactive systems are popular. The system uses retrieval functions to retrieve relevant answers from a document collection or knowledge base. However, developing RL-based interactive systems is not always successful. Prior methods either failed to build representations that provide a global picture of the task or could not enable the system to control the retrieval results directly. The costly labeling process of interactive data further handicaps the application of RL-based methods. The RL agents trained on limited annotated data may fail to generalize. The evaluation metrics for interactive systems are often unbounded, and the huge variance among search tasks may bias the evaluation.

In this dissertation, I formulate the task of building retrieval-based interactive systems as an RL problem and propose a general framework for building, training, generalizing, and evaluating RL-based interactive systems. I propose to present the system with a global representation of the knowledge repository to enable the full exploration in state and action space. I then employ a differentiable retrieval action to allow the system to control the retrieval process effectively. To improve the generalizability, I propose methods that adaptively train the system in randomized environments and generate high-quality, diverse interactions. I also propose a metric normalization schema that effectively improves the fairness of evaluation.