Skip to content

Home

Last updated: August 31, 2023

See the official TREC iKAT repository for tools and code related to the track

Introduction to TREC Interactive Knowledge Assistance Track

Voice-based assistant interactions are now widespread, with a recent Comscore report showing that over 20% of homes in America own a smart speaker. Furthermore, the recent announcement of assistant-enabled smart glasses from leading tech companies continues this trend towards real-world interaction. However, despite current assistants' ability to perform well-defined simple actions, their ability to support information seeking in conversations continues to be limited.

Conversational Information Seeking (CIS) is an established and important research direction. It is of interest to the broad research community within information retrieval, such as ranking, summarizing and question answering (QA), as well as for natural language processing and dialogue systems communities.

The TREC Interactive Knowledge Assistance Track (iKAT) builds on the experience of four successful years of TREC Conversational Assistance Track (CAsT), where the key focus of iKAT is on researching and developing collaborative information seeking conversational agents which can tailor and personalize their response based on what they learn about and from the user.

The fourth year of CAsT aimed to add more conversational elements to the interaction streams, by introducing mixed initiatives (clarifications, and suggestions) to create multi-path, multi-turn conversations for each topic. TREC iKAT evolves CAsT into a new track to signal this new trajectory. iKAT aims to focus on supporting multi-path, multi-turn, multi-perspective conversations, i.e., for a given topic, the direction and the conversation that evolves depends not only on the prior responses but also on the user (and their background/perspective/context/etc). As different personas undertake various topics, systems need to build and develop a picture of who the user is, in order to best address their information needs. Put another way, iKAT focuses on system understanding of user knowledge and information needs in accordance to the available context.

Shield: CC BY-SA 4.0

All data associated with this work is licensed and released under a Creative Commons Attribution-ShareAlike 4.0 International License.

CC BY-SA 4.0

Track Coordinators

Mohammad Aliannejadi, University of Amsterdam, The Netherlands. Dr. Aliannejadi is an Assistant Professor at the IRLab (formerly known as ILPS), the University of Amsterdam in The Netherlands. His research is in modeling user information needs with a focus on recommender systems, unified (meta) search, and conversational systems.

Zahra Abbasiantaeb, University of Amsterdam, The Netherlands. Zahra is a Ph.D. student at the IRLab supervised by Dr. Aliannejadi. She is working on conversational search and recommendation. Earlier, she has also worked on patent reference mining. Zahra obtained her masters in AI from the Amirkabir University of Technology with a focus on Question Answering systems.

Shubham Chatterjee, University of Glasgow, Scotland. Dr. Chatterjee is a Research Associate in the Glasgow Representation and Information Learning (GRILL) Lab, part of the Glasgow Information Retrieval group. The goal of his research is to design intelligent search systems which would one day respond to a user's open-ended and complex information needs with a complete answer instead of a ranked list of results.

Jeff Dalton, University of Glasgow, Scotland. Dr. Dalton is a Senior Lecturer (Associate Professor) at the Department of Computing Science, University of Glasgow. He is also the PI for the GRILL Lab. His research focuses on new methods for machine understanding of language and text data using deep neural networks and entity knowledge graphs for improving information seeking applications.

Leif Azzopardi, University of Strathclyde, Scotland. Dr. Azzopardi is an Associate Professor in Artifical Intelligence and Data Science within the Departement of Computer and Information Sciences at the University of Strathclyde. He is the PI for the Interaction Lab (i-lab) which specializes in developing, evaluating and modelling information rich and information intensive applications and agents.

Submit Your Runs

The submission system is now live! Before submission, please validate your run using the provided validation code.

Announcements

  • New! Baselines released. Guidelines updated. Demo released.

  • Additional data released for the participants. See the data section.

  • Train topics released! See the data section of the website.

  • Sample topics released! See the data section of the website.

  • Guidelines released!

Contact