"Can You Tell Me?": Designing Copilots to Support Human Judgement in Online Information Seeking

Creators: Bink, Markus and Risius, Marten and Elsweiler, David and Kruschwitz, Udo
Title: "Can You Tell Me?": Designing Copilots to Support Human Judgement in Online Information Seeking
Item Type: Conference or Workshop Item
Event Title: (Proceedings of the) 2026 Conference on Human Information Interaction and Retrieval (CHIIR)
Event Location: Seattle, WA, USA
Event Dates: March, 22-26, 2026
Projects: IDI
Page Range: pp. 172-182
Paper No. / Type of Paper: / Research Article
Additional Information: Open Access
Date: 2026
Divisions: Informationsmanagement
Abstract (ENG): Generative AI (GenAI) tools are transforming information seeking, but their fluent, authoritative responses risk overreliance and discourage independent verification and reasoning. Rather than replacing the cognitive work of users, GenAI systems should be designed to support and scaffold it. Therefore, this paper introduces an LLM-based conversational copilot designed to scaffold information evaluation rather than provide answers and foster digital literacy skills. In a pre-registered, randomised controlled trial (N=261) examining three interface conditions including a chat-based copilot, our mixed-methods analysis reveals that users engaged deeply with the copilot, demonstrating metacognitive reflection. However, the copilot did not significantly improve answer correctness or search engagement, largely due to a "time-on-chat vs. exploration" trade-off and users’ bias toward positive information. Qualitative findings reveal tension between the copilot’s Socratic approach and users’ desire for efficiency. These results highlight both the promise and pitfalls of pedagogical copilots, and we outline design pathways to reconcile literacy goals with efficiency demands.
Forthcoming: No
Language: English
Link eMedia: Download
Citation:

Bink, Markus and Risius, Marten and Elsweiler, David and Kruschwitz, Udo (2026) "Can You Tell Me?": Designing Copilots to Support Human Judgement in Online Information Seeking. In: (Proceedings of the) 2026 Conference on Human Information Interaction and Retrieval (CHIIR), March, 22-26, 2026, Seattle, WA, USA, pp. 172-182, Paper / Research Article. ISBN 9798400724145

Actions for admins (login required)

View Item in edit mode (academic staff only) View Item in edit mode (academic staff only)