Skip to content

Home

Last updated: April 9, 2025

See the official TREC iKAT repository for tools and code related to the track

About TREC Interactive Knowledge Assistance Track (iKAT)

The widespread adoption of voice-based assistants is significantly changing how we interact with technology. According to a Comscore report, over 20% of U.S. households now own a smart speaker. This trend is further exemplified by the recent introduction of assistant-enabled smart glasses by major tech companies, pushing the boundaries of real-world applications.

Despite their proficiency in executing simple, well-defined tasks, these assistants still face limitations in supporting conversational information seeking (CIS). CIS is crucial within fields such as information retrieval, natural language processing, and dialogue systems, focusing on tasks like ranking, summarizing, and question answering.

The TREC Interactive Knowledge Assistance Track (iKAT) builds on the four years of success of the TREC Conversational Assistance Track (CAsT), which can be explored further here. iKAT is designed to research and develop conversational agents that excel in collaborative information seeking by personalizing responses based on user-derived insights.

CAsT's fourth year introduced more interactive elements, such as clarifications and suggestions, fostering multi-turn, multi-path conversations. iKAT evolves from CAsT with a renewed focus on supporting diverse, multi-turn conversations tailored to the user’s background, perspective, and context. This means that for any given topic, the flow and substance of the conversation can vary significantly depending on the user’s individual traits and needs.

iKAT's primary goal is to advance research on conversational agents that not only respond to users’ immediate queries but also adapt their responses based on the cumulative context of the interaction. This aspect of personalization is particularly timely with the advancements in large language models (LLMs), which introduce new challenges and opportunities in the dynamic interplay of user context, system promptings, and conversational initiatives.

Shield: CC BY-SA 4.0

All data associated with this work is licensed and released under a Creative Commons Attribution-ShareAlike 4.0 International License.

CC BY-SA 4.0

Track Coordinators

Mohammad Aliannejadi, University of Amsterdam, The Netherlands. Dr. Aliannejadi is an Assistant Professor at the IRLab (formerly known as ILPS), the University of Amsterdam in The Netherlands. His research is in modeling user information needs with a focus on recommender systems, unified (meta) search, and conversational systems.

Zahra Abbasiantaeb, University of Amsterdam, The Netherlands. Zahra is a Ph.D. student at the IRLab supervised by Dr. Aliannejadi. She is working on conversational search and recommendation. Earlier, she has also worked on patent reference mining. Zahra obtained her masters in AI from the Amirkabir University of Technology with a focus on question answering systems.

Simon Lupart, University of Amsterdam, The Netherlands. Simon is a Ph.D. student at the IRLab supervised by Dr. Aliannejadi and Prof. Kanoulas. He worked in IR for the past two years at Naver Labs Europe, and joined UvA to focus on conversational search.

Nailia Mirzakhmedova, Bauhaus-Universität Weimar, Germany. Nailia is a PhD student at the chair of Intelligent Information Systems supervised by Prof. Dr. Benno Stein. Her research focuses on computational argumentation, framing, and user simulation for the evaluation of conversational search systems.

Marcel Gohsen, Bauhaus-Universität Weimar, Germany. Marcel is a PhD student at the chair of Intelligent Information Systems supervised by Prof. Dr. Benno Stein. His research concentrates on the intersection between information retrieval and natural language processing with a particular focus on conversational search, user simulation, and generative IR.

Johannes Kiesel, GESIS - Leibniz Institute for the Social Sciences, Cologne, Germany. Dr. Kiesel is the leader of the Big Data Analytics team at GESIS - Leibniz Institute for the Social Sciences in Cologne, Germany. His research interests include conversational search, argumentation systems, and user simulation.

Submit Your Runs (iKAT 2025)

We will share details of how to submit the runs soon. We will also provide a validation script to validate your runs for submission. Runs failing the validation script will not be accepted.

Announcements

  • Release of a questionnaire to get feedback on the used document collection.

Feedback

We hope to get your opinion about the document collection used in iKAT 2025. In the following Google Form you can tell us your opinion which should not take more than 2 minutes of your time. Your input is highly appreciated and can be vital in decision-making of iKAT.

Contact