Matilde Basilicati, Merle Beaujon, Jelena Prokic, Stephan Raaijmakers and Daan Vos
In real life conversations, humans continuously adapt their communication style, adjusting their language based on (a.o.) feedback, topics and estimates of linguistic capabilities of their interlocutor. Typical stylistic adaptation consists of variations of lexical choices, variable syntactic complexity or alternating speech rates. Under cooperative scenarios, stylistic adaptivity should contribute to mutual understanding. While natural to humans, such adaptive behavior is still largely lacking in current conversational AI, where a person communicates with an automatic system like a chatbot (or conversational agent). In our research, we attempt to partially bridge this gap by automatically identifying the linguistic complexity level of an interlocutor, using a range of linguistic features that reveal linguistic complexity in text-based interaction. The effectiveness of the proposed features is measured by experiments on Dutch data from Wikipedia, the Wablieft corpus and a proprietary human-AI interaction corpus. Our linguistic complexity detection allows a conversational agent to switch between low- and high complexity registers. In our research, we address the implementation of such an agent, specific challenges dealing with (vice versa) human adaptation to conversational agents, and the issue of measuring complexity in short texts.