Skip to content

Home

Last updated: August 5, 2024

See the official TREC iKAT repository for tools and code related to the track

Introduction to TREC Interactive Knowledge Assistance Track

The widespread adoption of voice-based assistants is significantly changing how we interact with technology. According to a Comscore report, over 20% of U.S. households now own a smart speaker. This trend is further exemplified by the recent introduction of assistant-enabled smart glasses by major tech companies, pushing the boundaries of real-world applications.

Despite their proficiency in executing simple, well-defined tasks, these assistants still face limitations in supporting conversational information seeking (CIS). CIS is crucial within fields such as information retrieval, natural language processing, and dialogue systems, focusing on tasks like ranking, summarizing, and question answering.

The TREC Interactive Knowledge Assistance Track (iKAT) builds on the four years of success of the TREC Conversational Assistance Track (CAsT), which can be explored further here. iKAT is designed to research and develop conversational agents that excel in collaborative information seeking by personalizing responses based on user-derived insights.

CAsT's fourth year introduced more interactive elements, such as clarifications and suggestions, fostering multi-turn, multi-path conversations. iKAT evolves from CAsT with a renewed focus on supporting diverse, multi-turn conversations tailored to the user’s background, perspective, and context. This means that for any given topic, the flow and substance of the conversation can vary significantly depending on the user’s individual traits and needs.

iKAT's primary goal is to advance research on conversational agents that not only respond to users’ immediate queries but also adapt their responses based on the cumulative context of the interaction. This aspect of personalization is particularly timely with the advancements in large language models (LLMs), which introduce new challenges and opportunities in the dynamic interplay of user context, system promptings, and conversational initiatives.

Shield: CC BY-SA 4.0

All data associated with this work is licensed and released under a Creative Commons Attribution-ShareAlike 4.0 International License.

CC BY-SA 4.0

Track Coordinators

Mohammad Aliannejadi, University of Amsterdam, The Netherlands. Dr. Aliannejadi is an Assistant Professor at the IRLab (formerly known as ILPS), the University of Amsterdam in The Netherlands. His research is in modeling user information needs with a focus on recommender systems, unified (meta) search, and conversational systems.

Zahra Abbasiantaeb, University of Amsterdam, The Netherlands. Zahra is a Ph.D. student at the IRLab supervised by Dr. Aliannejadi. She is working on conversational search and recommendation. Earlier, she has also worked on patent reference mining. Zahra obtained her masters in AI from the Amirkabir University of Technology with a focus on Question Answering systems.

Simon Lupart, University of Amsterdam, The Netherlands. Simon is a Ph.D. student at the IRLab supervised by Dr. Aliannejadi and Prof. Kanoulas. He worked in IR for the past two years at Naver Labs Europe, and joined UvA to focus on Conversational search.

Shubham Chatterjee, Missouri University of Science and Technology, Missouri, USA. Dr. Chatterjee is an Assistant Professor of Computer Science.Dr. Chatterjee works on Neural IR, Entity-Orinted Search, Conversational IR, and the applications for LLMs to these areas.

Jeff Dalton, University of Edinburgh, Scotland. Dr. Dalton is a Associate Professor (Reader) and Chancellor's Fellow at the School of Informatics, the University of Edinburgh. He is also a Turing AI Fellow and PI for the GRILL Lab. His research focuses on new methods for machine understanding of language and text data using deep neural networks and entity knowledge graphs for improving information seeking applications.

Leif Azzopardi, University of Strathclyde, Scotland. Dr. Azzopardi is an Associate Professor in Artifical Intelligence and Data Science within the Departement of Computer and Information Sciences at the University of Strathclyde. He is the PI for the Interaction Lab (i-lab) which specializes in developing, evaluating and modelling information rich and information intensive applications and agents.

Submit Your Runs (iKAT 2024)

We will share details of how to submit the runs soon. We will also provide a validation script to validate your runs for submission. Runs failing the validation script will not be accepted.

Announcements

  • 💥 New! Release of the TREC iKAT 2024 Test Topics.
  • Release of the TREC iKAT 2024 Guidelines.

Publications

Contact