arXiv Analytics

Sign in

arXiv:1808.07244 [cs.CL]AbstractReferencesReviewsResources

Improving Matching Models with Contextualized Word Representations for Multi-turn Response Selection in Retrieval-based Chatbots

Chongyang Tao, Wei Wu, Can Xu, Yansong Feng, Dongyan Zhao, Rui Yan

Published 2018-08-22Version 1

We consider matching with pre-trained contextualized word vectors for multi-turn response selection in retrieval-based chatbots. When directly applied to the task, state-of-the-art models, such as CoVe and ELMo, do not work as well as they do on other tasks, due to the hierarchical nature, casual language, and domain-specific word use of conversations. To tackle the challenges, we propose pre-training a sentence-level and a session-level contextualized word vectors by learning a dialogue generation model from large-scale human-human conversations with a hierarchical encoder-decoder architecture. The two levels of vectors are then integrated into the input layer and the output layer of a matching model respectively. Experimental results on two benchmark datasets indicate that the proposed contextualized word vectors can significantly and consistently improve the performance of existing matching models for response selection.

Related articles: Most relevant | Search more
arXiv:2004.03588 [cs.CL] (Published 2020-04-07)
Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
arXiv:2205.11113 [cs.CL] (Published 2022-05-23)
What Drives the Use of Metaphorical Language? Negative Insights from Abstractness, Affect, Discourse Coherence and Contextualized Word Representations
arXiv:1906.07337 [cs.CL] (Published 2019-06-18)
Measuring Bias in Contextualized Word Representations